Sitescooper Read over 400 news websites on your Palm handheld

There are a few similar projects out there: The big one is AvantGo, which does a similar thing, but with some major differences. First of all, the scooping occurs at sync-time (which I find slow); it's a big application, both on hotsync-side and palm-side; it does not support all the cool tricks sitescooper does to limit what content gets transferred; and it cannot track what content you've already read. Having said all that, they were here first, and I probably would not have written sitescooper if they had a Linux version when I started working on it!

The very first scooping-style tool, apparently, was InfoRover, which uses a similar snarfing mechanism to AvantGo -- it can scoop pages to a certain depth, and doesn't cut out redundant HTML. It requires a Windows machine to run, but is free for personal, non-commercial use.

Here's The Pilot Newspaper, which is a CGI script which parcels up news headlines from Yahoo, Reuters and some other sites then converts them to DOC format for Palm handhelds. I was planning to do something similar with sitescooper, but kind of got diverted and did the installable PRC files page instead!

Carsten Clasohm, who has provided lots of sites and contributions to sitescooper, is also the author of LeanWeb, a GPL'ed tool which transforms web pages in a similar manner to sitescooper's StoryStart, StoryEnd, and Pre- and PostProcess directives. It acts as a web proxy, and Carsten says he used it with "the old version of AvantGo".

In addition, Plucker is a free (GPLed) HTML viewer for the Palm, which can also display images and traverse links to a certain depth, providing scooping functionality in a similar manner to AvantGo. It, and its conduits currently runs on Linux, Windows, OS/2 and PalmOS 3.x. I plan to add support for it at some stage.

For German users, Sven Hauptmann operates a site called Palmtop Portal which seems, to this non-German speaker, to be a list of various German news sites, delivered as channels for iSiloWeb, AvantGo, InfoRover and the Pendragon browser.