Jonesieblog

In this series of posts, I’m going to show how I am customising the excellent “Planet Reader” application to produce a stand-alone program that updates data about a collection of blogs stored in a Ruby on Rails application - the feeds table looks like this:

create_table “feeds” do |b|
b.column “link”, :string
b.column “title”, :string
b.column “description”, :string
b.column “pubDate”, :datetime
b.column “language”, :string
b.column “error_tag”, :integer
b.column “site_url”, :string
end

Here’s how Planet Reader works now:

1. get “planet” information and a list of feeds from a config.ini file
2. create a “planet” object based on the config.ini data
3. visit each of the feeds using the Universal Feed Parser
4. If the feed has changed since the last visit, update the information stored in a cache file
5. “subscribe” the “planet” object to this feed
6. churn out static html files based on all the data.

For my purposes, the “planet” object is irrelevant, but I’ll leave it alone, as it helps the whole thing to hang together! My strategy for customising Planet Reader is as follows:

1. get database information from database.yml
2. connect to database and grab a list of all the feeds
3. export this list to a config.ini file
4. get “planet” information and a list of feeds from that config.ini file
5. create a “planet” object based on the config.ini data defaults
6. visit each of the feeds using the Universal Feed Parser
7. If the feed has changed since the last visit, update the information stored in a cache file and also update the database
8. “subscribe” the “planet” object to this feed
9. churn out static html files based on all the data.

Simple eh? Unfortunately I’ve never done any serious Python programming, so I’m learning as I go! To begin with, I’ll be editing the “planet.py” file, which is the file that gets executed to run Plant Reader. You can download the files here if you want to play along, and will, of course, need to have installed Python! I’ve been using the Eric IDE, which is a simple “apt-get install” on Debian.

Getting database information from database.yml

A bit of googling turned up PyYaml, which does the job. Once I’ve installed that, I just need to add:

import yaml
environment=”development”
dbconfig=open(”../config/database.yml”,’r') # we are in lib directory
config_data=yaml.load(dbconfig.read())
host=config_data[environment]['host']
username=config_data[environment]['username']
passwd=config_data[environment]['password'] or “”
database=config_data[environment]['database']

Getting a list of feeds from the database

Now I just need to connect to the database and pull out a list of feeds. First, install the Python-MySQL library. Planet Reader identifies feeds by their URLs, but of course my database will have an id field for each feed, so we’ll pull that out too:

Con= MySQLdb.connect(host,username,passwd,database)
Cur=Con.cursor()
Cur.execute(”select id, link from feeds order by ‘id’ “)
Results=Cur.fetchall()

Putting the feed data into a config file

This turns out to be pretty easy. The ConfigParser library has already been imported, and it does writing as well as reading. The variable config_file has already been set to default at “config.ini”, so:

config=ConfigParser()
for f in Results:
  config.add_section(f[1])
  config.set(f[1],”id”,f[0])
  output=open(config_file,’w')
  config.write(output)

does the job!

I wrap all this stuff into an “update_config_from_database” method, comment out the bit of code that insists upon some “planet” information in “config.ini” and I end up with Planet.py

Running this will produce a set of cache files and static html pages based on all the feeds in my database.

This is pretty cool, but I haven’t written anything back to the database yet. That’s what “Part 2″ will be about!

Stumble it!

We all know that we can do everything that Glow does so far with free tools available on-line.  That fact has no bearing, in my opinion, on whether or not Glow is a worthwhile enterprise, and so I shall refrain from banging on about it.

I’ve been back in the Glow portal this evening.  Everything worked smoothly on Linux running Firefox.  This is a real credit to the original architects of Glow. It would have been so easy to get sucked into a Windows or Windows/Mac only solution.

Once inside I began trying to set up a glow chatroom.  I have to say that this was not an intuitive process, but I got there in the end, and I’m sure it’ll be easier next time.

It did strike me, as I looked around, that this whole environment will by default be hidden to the wider world.  We will have to think carefully about the implications of that.  I currently have two classes doing scribe post blogs, and their global audience is a big selling point.  It will be a shame if the wider world isn’t able to see what we are doing or share the resources we create.

The VLE should, of course, have some privacy - I look forward to seeing how this turns out. Or I could just use Moodle now… oops, I promised didn’t I?

Stumble it!

I haven’t noticed anyone else in our little corner of the blogosphere talking about this, so here goes…

Tomorrow is shutdown day!  Can you spend a full 24 hours without a computer?

Stumble it!

I just read this at the clacksblog

News has just reached us that Mentor training is being delayed until after the summer holidays.

I wonder if this applies nationally? Anyone able to fill me in on the details?

[UPDATE] Apparently it does apply to us all.  No reason to panic - IT projects get delayed all the time. 

Stumble it!

I’ve been pondering ways to implement some of the remaining planned features in  scotedublogs.org.uk.

To this end, I’m wondering  - how many folk in the scotedublogs world have Technorati profiles, and have claimed their blogs with Technorati?  I’m rather hoping that the answer will be “most of us”, because I would like to integrate information from Technorati into scotedublogs.org.uk rather than asking bloggers to replicate information they have already submitted to Technorati.  Please vote and let me know

Stumble it!