Josh Huckabee

There are some very trivial administration tasks that I find myself doing over and over again in Drupal which can't currently be done from the admin interface. As of right now, these include installing a theme or module from drupal.org or adding a new site to an existing Drupal install. So, I've built a tiny Capistrano file that allows me to automate these tasks.

The tasks that I've setup are:

cap drupal:install:theme
cap drupal:install:module
cap drupal:install:site

cap drupal:install:theme

All this task requires is a path to the tar file of the theme that you want to install. What I do is browse over to the list of Drupal themes, find one that I want to try out, and copy the location of the "Download" link for the correct theme. In Firefox and Safari its as simple as right clicking the link and selecting "Copy Link Location". Make sure the version of the theme corresponds to the version of Drupal you have installed.

Once you've got the location to your theme copied, run the task with cap drupal:install:theme. It will ask you for the path to the theme location which you can then paste in. That's it. You should be able to browse to admin/build/themes and see your newly installed theme.

IBM recently published an article on creating spiders with Linux. I've always had a fascination with how this was done and knew it couldn't be that difficult. Reading this article does, however, make you appreciate the logic that has gone into the big Search Engines' spiders.

After a uniquely timed request to scrape some data from the web, it was like God was telling me, "Now is the time to learn web scraping and spidering." So, I jumped at the opportunity.

Fortunately, the first few examples in the article used Ruby. The last one used Python because of its built in HTML Parsing library. I was able to find an HTML parsing gem for Ruby called hpricot which I found to work very well.

Unfortunately, the request that I had to do this came about from a confidential project - so I can't share my code at this point. Nonetheless, it was quite a fun little project that included using ActiveRecord from Rails to store my scraped data in a MySQL database.

Check out the article and have fun programming. I know I did. Oh yeah, I looked at MySpace's robots.txt file and it only specified limits for a user agent that I couldn't recognize. I guess I should do a little digging to see if my scraper violates any legal terms of service.