Technical Inspirations

Reading RSS (XML) feeds in PHP5 - Part 2

2008-04-21T10:38:00.015+01:00

After adding my blog as a news feed last week, I decided it would be much tidier to restrict the number of blog entries on each page . This should reduce time to render the page, and most blog entries that are read will be the more recent ones anyway. The inclusion of Next/Previous buttons would still allow the reader to navigate through other blog posts.

Clearly, the page request would have to be processed seperately for each batch of posts, so this requires some changes to the page that calls the underlying function. The new inline page PHP code looks like this:

$nEntriesPerPage = 5;
$blogIndex = (int) $_GET["index"];
$foo = doShowBlogSections($blogs[0], $blogIndex, $nEntriesPerPage); 

The number of entries to be show per page is 5 and by casting the index to an integer, if it isnt passed through by the page, we default it to zero.
Next Up is the function itself, the doShowBlogSections call does this:

function doShowBlogSections($blogURL, $blogEntryIndex, $nBlogEntries) {
 // get rss feed contents
 $library = @simplexml_load_file($blogURL);
 $i=0;

 // loop over each entry and output if relevant and possible
 $needNextButton = false;
 foreach ($library->entry as $entry) {
  // only show nBlogEntries...if we exit the loop normally, no next button required
  if ( ($i) == ($nBlogEntries+$blogEntryIndex) ) {
     $needNextButton = true; // having to bail - there are more entries available
     break;
  }
  // only output blog entries from blogEntryIndex onwards
  if ($i >= $blogEntryIndex ) {
     echo "<p>";
     echo "Title: ".$entry->title."<br />\n";
     $pubDate = $entry->published;
     // sort out obscure date format used by blogger.com
     if ( $pubDate = strtotime($pubDate) ) {
        // we have a valid unix timestamp value from the date string
        $pubDate = date("F j, Y, g:i a", $pubDate);
        echo "Published on ".$pubDate."</p>\n";
        }
        else {
        // give up and just output in whatever format the date/time is in
        echo "Published on ".$entry->published."</p>\n";
        }
        echo "<div>".$entry->content."</div>\n";
  }
  $i++;
 }

 // Provide suitable Next and Previous buttons
 if ( $blogEntryIndex == 0 ) $needPreviousButton = false;
 else $needPreviousButton = true;
 $blogPreviousIndex = $blogEntryIndex - 5;
 if ($blogPreviousIndex < 0 ) $blogPreviousIndex = 0;
 $blogNextIndex = $i;
 
 $thisPage =  $_SERVER['PHP_SELF']; // need to be self referencing for page name
 echo "<p>";
 if ($needPreviousButton) echo "<a href=".$thisPage."?index=".$blogPreviousIndex."><<< Previous</a>";
 if ($needPreviousButton && $needNextButton) echo '      ';
 if ($needNextButton) echo "<a href=".$thisPage."?index=".$blogNextIndex.">Next >>></a>";
 echo "</p>";
 
}

It should be fairly self explanatory as the code has a liberal sprinkling of comments. Other enhancements that I may add at a a later date include:

First and Last buttons

Some indication of total number of log entries and where you are in that list

a list of blog entry titles and dates as clickable links

a link to the comment page

Reading RSS (XML) feeds in PHP5 - Part 1

2008-04-18T11:07:00.006+01:00

Another new addition to the website, is the reading in of RSS feeds from my blogs. This seemed much harder than it actually was, especially was I got my head around the built in PHP5 XML functions. In essence, you just call @simplexml_load_file("the_xml_url") and the loop over each entry. The hardest part is working out what the section names are within the xml file, and you can just load the XML into a text editor to find out. The code I am using goes as follows:

function doSimpleShowBlog($blogURL) {
  // get rss feed contents
  $library = @simplexml_load_file($blogURL);
  // loop over each entry
  foreach ($library->entry as $entry) { 
   echo "<p>";
   echo "Title: ".$entry->title."<br />\n";
   echo "Published: ".$entry->published."</p>\n";
   echo "<div>".$entry->content."</div>\n";
  }
}

Simple huh! Some error checking and so on is recommended but this works for me. An example can be found here.

New website

2008-04-15T19:58:00.001+01:00

Well, my new website is up and running (www.birkbeck.org.uk) using a modified version of the Satellite v1.5 PHP scripts. As time goes on, I will be adding functionality - including feeds from this and other blogs of mine - and finally taking some time out to learn some PHP.

Most recently, I have been working to get a Flickr Tag Cloud working in a way I like. The original scaled the font according to the number of occurrences of each tag, but the distribution of occurrences was very wide - for instance, most images are tagged with England, Britain and UK. The order of magnitude difference between the most used tags and the majority of tags is very large and the way it had been coded meant that only those tags used in approx 70% or more of the photos in my stream were shown in anything other than the smallest font. The code below is a snippet I have added to calculate font sizes based on a normalised distribution of occurrences.

// Calc font size for a normalised distribution........
// First build array of total counts for each tag count
$i = 0;
foreach ($theCount as $count) { $tagCounter[$i] = $count; $i++; }
// sort array of counts
sort($tagCounter);
// Remove duplicates
$sortedTagCounts = array_unique($tagCounter);
// create an array of preset font sizes determine by tag count
$totalTags = count($sortedTagCounts);
$fontRange = $maxFontSize - $minFontSize;
$fontSizeMultiplier = (($fontRange-$minFontSize+1)/$totalTags);
$i=0;
foreach ($sortedTagCounts as $tagOccurrence) {
 $tagOccurrencesMappedToFontSizes[$tagOccurrence] = $i*$fontSizeMultiplier)+$minFontSize;
 $i++;
}

In essence, I create a sorted list of tag occurrence frequencies, e.g. one tag may occur 8 times, another may be used over 1000 times. In all, there may be 50 different occurrence numbers. If two different tags are used 10 times each, then this is one entry in the list. Then, I created an array based on each number of occurrences that holds the font size.

Later in the code (not shown here), when I loop through the tags I just use the occurrence frequency as an index into the array of font sizes. There may be easier ways to accomplish this, but I am reasonable happy with it, particularly given my lack of knowledge regarding PHP syntax and built-in functions.

I am working on a revised version of the code and plan to release the full source code, once I have added integral blog pages, but first need to write some PHP to feed that in via XML and RSS. In the meantime, if you want a copy of the code, drop me an email.

Xorg config for Iiyama Vision Master 1451 under Ubuntu

2007-07-28T19:29:00.000+01:00

As Ubuntu did not recognise this by default, it requires some jiggery pokery with the X config. Before editing, it is recommended that you make a backup of the file /etc/X11/xorg.conf, then modify the Monitor section as follows:

Section "Monitor"
Identifier "iiyama Vision Master 1451"
Option "DPMS"
HorizSync 28-80
VertRefresh 75-75
EndSection

...and also change the Monitor value in the Screen section to match the Identifier, in this case to be "iiyama Vision Master 1451"

Ubuntu performance improvements

2007-07-25T18:31:00.000+01:00

If you have 1GB of memory of more, reduce the virtual memory usage....
$ sudo cat /proc/sys/vm/swappiness
60
$ sudo sysctl -w vm.swappiness=10
vm.swappiness = 10

To make the change permanent
sudo gedit /etc/sysctl.conf
Add the above line (vm.swap...) to the end of the file and save

More useful performance stuff can be found at the Ubuntu Forums

Details on how to update your kernel to ensure you have one optimised for your processor can be found here.

Digital Image Management - and the winner is.......digiKam

2007-07-23T05:04:00.000+01:00

Nothing so far meets all my requirements. Picasa is a shade too slow and unwieldy running under WINE (admirable though it is to work at all!). F-Photo is even slower handling my RAW Nikon NEF files.

After lots of searching around I have settled on using digiKam, though it is touted as being for KDE, it seems to run fine under Gnome. There are a few niggles, including the viewer not rotating portrait orientation images when you zoom in and the built in image editor is a little quirky in comparison to Picasa (though it is more powerful). For any proper image editing I will fall back on using GIMP.

The package has a built in LightTable that I am still trying to understand and has the facility for exporting to jpg and uploading to Flickr (via jUploadr). It allows the images to be organised as they are found on disk - for me this means YYYY/YYYY-MM-DD-place-or-event, includes tagging, but seemingly doesnt allow images to be stacked.

Some of the above may be due to my lack of understanding as I am still on the beginning of the learning curve - it has quite a wealth of features.

Other required functions I will either script or knock something together using MonoDevelop - a good excuse to get up to speed on c# and the GTK# toolkit.

If you feel like installing it, follow instructions at http://www.digikam.org/?q=download/binary/

Note: To get the "Archive to CD" function working, I needed to "sudo apt-get k3b" - A CD/DVD burning packing (although I could possibly have changed the default burning application)

Image workflow and management

2007-07-19T21:09:00.000+01:00

Ok, USB problems resolved (kind of - in USB1.1 mode), it is time to start thinking about my digital image workflow and management. This will not be a quick fix, there is not a single solution that meets all of my requirements, but that is part of the motivation for moving to Ubuntu/Linux - to find the tools that do meet individual requirements if possible and then glue them all together somehow.

So, what do I need? Some of the things on my wishlist include...
1 - Automation of copying from USB device (camera or card reader) to main photo directory
- user defined parent location
- directory name from date time of import or shooting with optional location suffix
- bulk rename
2 - Quick Review
- ala picasa - an opportunity to quickly review and remove and "duds"
- permanent deletion
3 - Tagging
- of the full set or partial set, perhaps location names etc
- Rating system?
4 - Archive new photoset
- To CD/DVD, multiple copies
- Split across media?
- serialised numbering system for restoring?
- offsite/onsite copies
5 - Review and basic changes
- ala picasa, changes stored as text files
- more tagging?
- deletions - stored thumbs and changes if archive has been created otherwise warn
6 - Backup/Archive
- to network or usb attached storage
7 - Uploads to photosite
- plug in interface for different sites
- autotag ref number to allow location of original from photosite
- name - description - groups - tags
8 - Advanced editing
- start 3rd party app
- somehow track changes (stacked photos?)
9 - Storing changes
- .myapp sub directory inside each photo folder
- individual text files for each set and for each image
- thumbs - 160x100 jpgs - even of archived
- ability to access work from multiple machines
- ability to work on either local or network copy and two way sync the changes

Hmmm, I dont want much then! There are some apps out there already, such as F-Spot, but having tried it, it is slow and doesnt really meet many of my requirements, though F-Spot is written in C# so there may be some opportunity to extend or change its behaviour and functionality.

I need to think about each part of the above in depth!

USB 2 bug?

2007-07-18T07:13:00.000+01:00

Damn and blast, just when I thought it was safe....copying bulk amounts of data to my external USB2 disk seems to be problematic. It seems that copying large files (meaning in the 5-100MB range) works ok but copying alot of smaller files causes some kind of buffer overrun and the drive disconnects. It looks like some kind of caching problem, not all drives or USB controllers are affected, obviously mine is.

Reports and dissection can be found here and a more recent bug report here

Useful commands:

dmesg | tail -20

lsusb

One workaround is to run the command "sudo modprobe -r ehci_hcd" to drop down to USB1.1 - so I will give that a try. There are a couple of other suggestions, such as hacking max_sectors or altering kernel details and recompiling, but if the modprobe works I will add it to my login script until an official fix becomes available.

Mounting NFS server at boot time

2007-07-17T19:28:00.000+01:00

Working on being able to access all my data from initial Ubuntu boot without any command line jiggery pokery. A quick edit of the disks to mount at boot time:
sudo vi /etc/fstab

Adding the following line at the bottom of the file
tux:/home /net nfs rw,soft,intr,rsize=8192 0 0

Firefox Addons

2007-07-17T17:18:00.000+01:00

Continuing the theme of configuring my new Ubuntu/Linux environment I find myself missing those Firefox addons that make life easier. A quick wander across to the official addon page yields

British dictionary (my spelling is actually pretty good, even if I do say so myself, but my typing is appalling)
GreaseMonkey - for all those handy flickr hacks
Del.icio.us - as I am hoping to have my bookmarks/favourites accessible anywhere

Next up is the Flickr Greasemonkey Hacks. This is a real pain, as whenever I setup a new PC, I have to go and grub around for the ones I use, even thought there isnt that many, at least if I list them here, I have something to refer to in the future....

Flickr Referer into Comments - adds the seen in text to your comments
Group Pool Admin: Warn + Delete - does pretty much what it says on the tin
Flickr Buddy Icon Reply v3 - to add the graphic of people you are replying to in comments and threads

If you have come across any other really useful stuff like this, let me know.

Useful unix command....so I dont forget...cp

2007-07-17T04:12:00.000+01:00

cp -uvr /home/photos/2007/* /media/EXTDISK/photos/2007/

I'm always forgetting the optimum syntax, must be my memory in my old age, but this one is likely to be used fairly frequently

Goodbye M$ Windows

2007-07-17T01:20:00.000+01:00

But Why?

Having given it some thought over several weeks about wiping my hard disk and installing a Linux variant, I had just about decided on Ubuntu when I received a USB2 external disk and PCI card which I failed miserably to get working under Windows XP, no matter how much shuffling of existing cards and resources that I did.

Not good timing though, with the local Middlesbrough Mela a day away. I had previously shot all photos in NEF (Nikon's RAW file format) and had a workflow that was fairly quick and simple based around Google's Picasa.

Yet I am getting ahead of myself. Why would I want to do this, really I mean. I have a legit installation of XP Pro, but certain things frustrate me, its time for a change and I really want something I can extend myself with scripts, tools, etc. My software development skills have fallen out of date and it is time to get up to speed with some of the more recent developments of the last few years. I also want better integration, from my desktop, internet, 3G mobile, of all the tools I use, email, calendering, contacts, bookmarks, the whole shebang. Maybe its a pipe dream? With all my data backed up on my Fedora Core 4 Linux server, email exported and bookmarks consigned to the bin, it is time to take the plunge.

Ubuntu Linux

Downloading and burning a copy of Ubuntu Feisty is fairly easy and straight forward. Installing should have been - a minimal set of questions, going with the defaults for most things, but life wasnt that simple. The partitioning was done within minutes, but make filesystem sat and hung at 5%, grrrrr. Several attempts later, I gave up on ext3 and changed the settings for an ext2 filesystem. Damn it is slow to create the filesystem, if you are doing this - allow several hours.

The good news is that the sound drivers, network card, USB interfaces and external disks all worked straight off the bat.

Nvidia Graphics Drivers

Slight problem-ette - screen resolution is poor, the install didnt seem to recognise my Nvidia GeForce3 card. No amount of "sudo apt-get install" seemed to bear fruit, but the light at the end of the tunnel was AutoMatix - which installed and configured the drivers for me as "Restricted Drivers" - phew! Without a decent screen resolution it would have been goodbye Ubuntu.

Other applications

I followed a few blog entries from others and installed additional applications with the built in Add/Remove tool where the packages where I could - trying to follow the rules as much as possible. For the rest, I turned to AutoMatix. What did I want that wasnt in the default install? Hmmm, things like - Opera, gFTP, BloGTK, Vmware Server, GnuCash, MonoDevelop, MP3 player (including codecs), SAMBA (for sharing disks and printers with other Windows machines like the kids), etc.

Photo Manipulation Stuff

Having been to the Mela (did I mention the install took a while), I had a bunch of pictures to process and the available in tools were just way too slow and tedious to use. This is no different to XP, but on Windows, there was always Picasa. I grubbed around a little and discovered that Picasa runs under WINE, so I could have the best of both worlds. I installed both and lo and behold, performance is about on par with XP. Excellent.

And for more in depth RAW file manipulation - I could use GIMP which comes preinstalled - right - erm - wrong actually, yes it comes preinstalled, but by default it doesnt handle raw files. Something about Tiff file errors and unrecognized tags! More grubbing around yielded a write up of UFRAW and the GIMP plugin. A magic incantation at a command line (terminal window) of "sudo apt-get install gimp-ufraw" fixed the problem.

Conclusion

I had read reports of Ubuntu that said that finally here is a version of Linux that is ready for the mainstream. I beg to differ. It is probably fine if it comes preinstalled and you dont need any extra applications or you are prepared to read up alot and get your head around some of the techie stuff (or have someone else to do this for you), but I wouldnt recommend it to non techie friends - I would probably spend half my life getting it working for them. Most people will have specific uses for their PC, and I dont believe it is simple enough to configure for most users.

I also havent found any preinstalled AntiVirus solution, but ClamAV is meant to be good so I will give that a look.

For me, it looks like it might fit the bill, time will tell, but I will try and post updates here.