Posted by & filed under Tech.

Dave Winer made this post on his rssCloud Blog today and it started the gears grinding. In it, he suggests the need for a “programmable” Twitter client. Specifically, he offered up the idea of a Twitter client with a scripting engine in it that would allow certain types of logic to be triggered on behalf of the Twitter user without having to involve Twitter or a reprogrammed client.

The example he used was wanting to un-follow a particular user for a day, then re-follow them automatically. It’s certainly possible to build this sort of functionality into an existing Twitter client, but I’d like to suggest something easier than herding all of the Twitter client authors in this direction. Specifically, rather than having the scripting support built into a Twitter client, why not just ask Twitter client authors to allow their clients to be pointed at alternate hosts that implement the Twitter APIs besides Twitter’s own servers?

This would allow us to implement Twitter “proxies”, so that even the most basic of Twitter clients could take advantage of all sorts of extra features and functionality that might get wrapped around those APIs. Using Dave’s example, here’s how it might work:

User configures their copy of UberTweet (a fictitious Twitter client) to use APIs at www.spiffyubertweet.com. Now when they use their client, all of the calls will pass through the Twitter proxy at www.spiffyubertweet.com, where all sorts of magic can happen. In the simplest implementation, the proxy just passes all calls to Twitter and all responses back to the calling client  with no mods or changes. The fun comes when the proxy starts paying attention to the content of the tweets.

Suppose that the user wants to un-follow “pottymouth” for 24 hours. Why not send a tweet like:

<unfollow pottymouth for 24 hours>

The proxy would catch that tweet, parse out the syntax inside the <>, and issue the necessary API calls to unfollow pottymouth. Then it would queue up a task for 24 hours later to re-follow pottymouth for the requesting user.

Twitter never sees the tweet. The client author had to make an utterly trivial modification to their client, and the user can suddenly reap the benefits of whatever services the proxy author chooses to insert into the pipeline. Proxies could allow all sorts of external meta data to be associated with a tweet, for example. Or you could envision Twitter proxies that performed all sorts of analysis on tweets and followers and return reports to users. The opportunities are endless and it requires virtually no work on either end of the existing Twitter ecosystem beyond allowing expert users to specify a Twitter API proxy in their Twitter client’s preferences.

There are several client-side scripting engines that support acting as both a client and a server and they are fully capable of implementing the Twitter APIs, even if most are only pass-through calls. The only thing holding this up is finding a popular Twitter client author that is interested in adding this tiny modification to their code. Any takers?

Posted by & filed under Tech.

The Java source code for rssNimbus (in the form of an Eclipse project snapshot) is now available from this link. Ultimately, it should be moved to an on-line version control site like Google Code, but for now a big blob is what you get. See the ReadMe.txt file inside the archive for more info.

You can find details on installing and running rssNimbus in this blog post. As a note to developers, you should use the Google Eclipse functionality to install your version of rssNimbus on Google App Engine instead of the command line tool as referenced in the blog post.

This source code is being released under the Creative Commons Attributed-Share Alike 3.0 license, which means you can pretty much do what you want with it as long as you give me credit and you make changes available under similar license terms.

Posted by & filed under Tech.

rssNimbus is an implementation of a rssCloud server using Google App Engine (GAE) to run the server’s Java code. To install and run your own copy, you need a few things:

  1. download a copy of rssNimbus. <http://www.shotton.com/files/rssNimbus.zip>
  2. create a Google App Engine account. <http://appengine.google.com/>
  3. download  the GAE SDK for Java. <http://code.google.com/appengine/downloads.html> or the current version directly from <http://googleappengine.googlecode.com/files/appengine-java-sdk-1.2.6.zip> (we need the “uploader” tool from this archive in a minute.)

Setting up GAE

The first thing you need to do is configure your GAE account so that you can upload a new application. Go to the GAE start page <http://appengine.google.com/> and click the “Create and Application” button.

Fill in a unique ID for your rssNimbus install (it must be all lower case and globally unique across all possible GAE apps) and give it some meaningful title and hit “Save”. Make note of the unique ID you created.

Installing rssNimbus

Extract the folder from the rssNimbus.zip archive. It will make a folder called “war”, which is the web archive containing all of the parts of rssNimbus that will be uploaded using a tool in the SDK archive you downloaded earlier.

IMPORTANT! You MUST edit one file in this folder in order to provide GAE with the unique ID of your app. Inside the “war” directory is a subdirectory called “WEB-INF”. In that directory is a file called “appengine-web.xml” (war/WEB-INF/appengine-web.xml) that you must edit with a text editor, changing the <application> value from “sampleUniqueID” to the actual value of your app’s unique ID.

The Google page at <http://code.google.com/appengine/docs/java/tools/uploadinganapp.html> describes in detail how to upload a new app. Here are specifics for doing this on Mac OS X or Linux from the command line.

– run the terminal application of your choice (Terminal.app on the Mac) and change your working directory to the directory containing the unarchived GAE SDK for Java and the “war” folder (e.g., cd ~/Downloads/ )

– assuming your “war” folder is also in the same location as the SDK, use the following command to upload your new GAE application:

For Mac OS X:
./appengine-java-sdk-1.2.6/bin/appcfg.sh update war

What Now?

You can now visit your GAE dashboard and confirm that the new application has been installed (current version should be “1”).

You can visit your rssNimbus server’s home page by pointing your browser at <http://yourUniqueID.appspot.com/>, substituting your actual unique ID in the URL.

You can use the forms on the rssNimbus home page to test pings and subscribe requests. There is a basic /admin page that also shows active publishers and subscribers.

You can get MUCH more detail from the GAE dashboard pages for your app, including access logs, database contents, queued tasks, etc.

Setting Expectations

There are a few things to look out for. The first pings and notifies to and from your server will take longer than you might expect as GAE sets up databases and task queues for your application. Subsequent requests will go much faster.

rssNimbus does NOT currently expire subscriptions. It deletes those that fail (in order to limit lengthy, CPU-expensive failed HTTP requests) immediately.

If you have questions or comments, please post them as comments here.

Posted by & filed under News, Tech.

The first demo deployment of rssNimbus, a rssCloud server for Google App Engine, is on the air for limited beta testing at http://rssnimbus.appspot.com/. I would greatly appreciate any feedback on how it works for people attempting basic pings and notifications.

Be forewarned that it still needs to be hooked into GAE’s cron processes so that hourly housekeeping can be performed (i.e., expiring subscriptions, pushing out delayed notifications, etc.)

Also, please note that as of this iteration, no subscriptions are being expired.

You can find a test version of rssNimbus at http://rssnimbus.appspot.com/ with additional instructions on its home page for using it. Please post any comments, issues, bugs, or feedback as comments to this post.

Once the final kinks are out of it and the basic UI is turned on for admin functions, a user-installable version of rssNimbus will be made available as a .zip archive for anyone who wants to take a shot at installing and running their own GAE-based rssCloud server. Hopefully that archive will be available this weekend, depending on feedback and last minute issues.