For a while I’ve been thinking it would be cool to scrape the Bureau of Meteorology rainfall images and make a movie out of them. The result was not quite as good as I imagined but interesting anyway. I’ll make a better one in the future with more data.
Digging through the page source I discovered they show six images in a cycle at a time. Images are removed from the loop location so I couldn’t just edit filenames and download the previous 1000 or so images – so I had to scrape them as they are used. A typical image looks like this:
Clearly that didn’t have all the data so I dug a bit further and found three other images that this is overlayed on top of to make the image you see when you visit the site. These images contain the basic map of the area, the topography, the city locations and the key legend for the rainfall density.
I used ImageMagick ‘composite’ tool to overlay them. You can only do two images at a time so I prepared the four underlayed images into one and then wrote a script with a column-editor to batch the scraped images onto that background.
To get all the images I wrote the scraper in PHP and set a cron task. Images are uploaded every six minutes (10 an hour), so I just had a cron task running every 30min to make sure I didn’t miss any.
*/30 * * * * root /usr/bin/php /home/pinky/bom-data/index.php
The actual script to scrape the images is:
And my script to merge each image onto the background layer looks like:
#!/bin/sh /usr/bin/composite DR023.T.201012030224.png background.png DR023.T.201012030224.png /usr/bin/composite DR023.T.201012030230.png background.png DR023.T.201012030230.png /usr/bin/composite DR023.T.201012030236.png background.png DR023.T.201012030236.png . . .
The nice time-stamped image names made everything easy. Overlaying 720 images took about a minute. I was surprised how long it took, even though each image is only 10kB or so.
To join all the images into a movie I used mencoder.
mencoder "mf://*.png" -mf fps=25 -ovc lavc -o output.avi
That just puts all the png’s in the directory into an encoded (compressed) AVI at 25fps. The resulting file was only 2.7MB and runs for 28 seconds. There wasn’t much rain in the last couple of days so it gets a bit boring to keep going further than that.
Here’s the resulting movie which shows the period 3rd Dec to 6th Dec.
Sorry for the very ordinary quality – I’d have to read up what encoding YouTube likes to make it nice because the source PNGs are very clear. Using mencoder with -lavc and then Windows Media Maker pretty much kills it.
To give credit to the soundtrack, the music is http://www.youtube.com/watch?v=aeYTBvanFmE
I’ll make a longer movie once I’ve collected a couple of months worth of data.