Phillip Pearson - web + electronics notes

tech notes and web hackery from a new zealander who was vaguely useful on the web back in 2002 (see: python community server, the blogging ecosystem, the new zealand coffee review, the internet topic exchange).

2006-1-8

Open-source infrastructure: Universal file uploader

Here's something I'd like to see: an open source project to produce client and server code to make it easy to upload files to websites.

[Update: we have a mailing list! Mail simplemediaupload-subscribe@topicexchange.com to join.]

It's not hard to add file/image upload support to a website, but some functions just work so much better on the client. If you can resize a 2MB JPEG down to 100K on the client, for example, you can send 100 photos over a 128K DSL line in 11 minutes rather than three and a half hours.

And... it seems that everything needs file/image upload support. Certainly all of our projects at work - Structured Blogging, OurMedia, GoingOn and PeopleAggregator, to name a few. Also a couple of my own personal projects, and pretty much every blogging tool. I think Marc has to resize images using Photoshop then upload them with FTP to get them on his blog. Wouldn't it be nice to be able to resize/crop inside a simple client and get the upload for free?

A few websites do this really well. Flickr's Uploadr is excellent. OurMedia users seem to have managed to upload quite a bit of audio and video with its uploader, so I assume it's not bad either. Then some sites don't use a client - Cyworld uses a Flash widget, which is OK, whereas MSN uses an ActiveX control that I completely failed to get to work on my neighbour's computer and haven't bothered to try at home. Outlook Express is good - it has an excellent little resize dialog that pops up when you e-mail image files.

... however, there doesn't seem to be a universal way to do this, that can be easily integrated into apps. So, who's gonna build it? :-)

- - -

Here's an initial stab at the work required:

  • A client app like Flickr's Uploadr that can:
    • Resize/convert images
    • Upload multiple images.
    • Upload larger files - e.g. videos and audio recordings.
  • The server side plumbing for this:
    • Some way of hooking this into a website - perhaps using a small chunk of XML that gives authentication credentials and an ID for the image to be uploaded.
    • Protocol to receive large files over unreliable connections, perhaps using multiple HTTP requests. WebDAV might suit here, if it's implementable in every brain-dead server environment.

Places to start to steal code for the client would be the FotoBilder client(s) and the OurMedia uploader.

- - -

The simplest way I can think of to hook this into apps would be to have an 'upload' link that pops up a Javascript window and sends you an XML file (with a mime type and defined extension - so it's easy to catch on the client side) which is picked up by the client. The XML file includes an authentication token and an image ID, which could be munged together into one opaque ID, and perhaps some parameters about what sort of files are acceptable (formats, sizes, number of files, ...). The client then makes HTTP requests (PUT requests, if you want to be fancy... perhaps a whole lot of them in series, for a large file, uploading 300K-1M or so at a time) back to the server to send all the files, sending back the auth token as it goes.

The Javascript window could poll the server for status, perhaps holding open a persistent connection. When the uploads are complete, the server would tell it about the received image and perhaps send back a URL of a thumbnail. The window would them close and the page requesting the image upload would be updated with the thumbnail and upload details.

Comments invited... is there an easier way to get the same functionality? Has someone already build this?

... more like this: []