Tagging Library for AppJet

The original and current versions of Vortices, have an only slightly more flexible category system than Reddit. On Reddit, you have to explicitly create a category, whereas Vortices had ad hoc categories with an option to explicitly create one. But this isn't quite what I had in mind. I want tagging, as Flickr and Delicious have, loose groupings that emerge naturally.

So, I built a tagging library that does exactly it. Then I built an example app to test it out. Surprisingly, it worked after I changed a few typos. What took the most time was all of the initial rewrites. I was having a hard time structuring it because I wanted to track who was doing the tagging and how many times an item was tagged with something, but in the end I decided it was too much effort as well as an impact on performance.

fbFund oddity

Facebook is having a contest for new app ideas on their platform, with huge grants awarded to the winners (more information). Oddly, there isn't even anything that says you have to actually make the app. But the strangest part wasn't even that. Reading the terms and conditions for the fbFund, an odd phrase jumped out in the middle of page 3:

(2) Effective Use of the Android Platform (17%)

So let me get this straight. Facebook copied Google's Android developer challenge and didn't even bother to change the branding? This is beyond lame.

Adding audio enclosures to feeds with Yahoo Pipes

I just finished my second "pipe" using Yahoo Pipes. If you haven't looked into Pipes, I strongly recommend it, because it is a testament of capabilities the web just didn't have until recently. It's main use is manipulating RSS feeds to create new feeds. Also, a pipe can make feeds from scraping web pages. Instead of editing source code as text, you insert graphical modules and fill in values to configure them. Then, whatever a module outputs can flow into the next module and the next, until finally the output is produced. Both of the pipes I created solve the same problem, adding audio enclosures to a music feed that was lacking them. I had to identify a pattern between the information in a feed entry and the URL of the MP3 file. The first one, Newgrounds weekly audio winners (original feed, my version, pipe page), was easy. All I had to do was copy the link and change "listen" to "download".



An experienced Pipes developer could make this in 2 minutes. But I have no idea what I'm doing, so it took me a couple hours of trial and error. The second one, remix.kwed.org (original feed, my version, pipe page), was a little more complicated. The link is http://remix.kwed.org/?search={id}. The MP3 is located at http://remix.kwed.org/{id}/{title}.mp3. Still not too bad, though, since all of the information was right in the feed, so no page scraping was necessary.

I still don't think I understand how to really use Pipes, as I am used to textual coding rather than visual. Maybe they will make it even easier to use in successive versions, assuming Yahoo isn't disbanded, with all of its cool apps sent to the scrap heap. But for now, if there's a quick hack you wish for involving feeds, and you don't feel like opening up a new programming project and figuring out how to host it, Pipes is your best bet.

AppJet databases now faster and more powerful.

Good morning, everyone. You might like to know that AppJet, the cloud-based JavaScript framework that nobody knows about, just released an update, making it more efficient for the CPU and easier on the coder, a win-win. There are new methods that make it easy to filter and paginate and data sets. See the AppJet change log for specifics.I have prepared an example to show the new features in action.

Suppose you have a collection, storage.threads and each object has a property, text, that you want to display. But it would be unreasonable to display every single thread at once, so some sort of pagination is in order. SQL has a handy LIMIT command, and now AppJet has an equivalent, skip() and limit().

//Pagination variables
var start = parseInt(request.params.start) || 0
var items = parseInt(request.params.items) || 10

The first step is to figure out where we're starting and how far to go, using request parameters found at the end of the URL. If not specified, we fall back to the front page (0) showing 10 items.

storage.threads
  .sortBy('-timestamp') //sort first so we skip the right ones
  .skip(start)
  .limit(items)
  .forEach(function(thread){
          printp(thread.text)
  })

Then we make the database query and print the output in one shot. In order for this to work, each thread has to have been given a timestamp property when it was created. The obvious thing to use is a Date object, which is really convenient because it easily casts itself to an integer when needed. So sortBy() sorts things from least to greatest, and since time always increases, it would sort in chronological order. All it takes to reverse this is prepending a minus sign to the argument. Now we simply skip anything before we want to start printing and limit the query to a certain number of items. It reads a lot like English. Finally, at the end we loop through the results of the query and print paragraph tags with the text inside.

You can see a more advanced version of this example running live on http://post.appjet.net/, with the source code available. Next time I'll show you how to generate the links for pagination.

Please Solve Comment Fragmentation

Friendfeed is a leap in communication. Using RSS and proprietary activity stream APIs, you can track what people are doing across the web instead of just on one site. Facebook has some inklings of this as well. Whoisi adds wiki functionality which is parallel to imaginary friends on Friendfeed, where you can follow an activity stream of someone who didn't sign up or import a certain stream. XMPP will make the stream real-time and less intensive on networks.

This is all well and good, but there is still a problem. Comments. For one, Friendfeed doesn't thread the comments, which makes it too hard to have a conversation. But even worse is the fragmentation. Other sites, such as Reddit, will maintain a single page of comments for any particular URL. But Friendfeed can create several, and there is no way to find them all. There could be the original import of the item (maybe a blog post or image on Flickr) from the content creator, followed by any number of others' links to that item from Google Reader, Twitter, Digg, and the rest. And when those actions appear on Friendfeed, they all have the potential to create yet another silo of comments. You may see something interesting and be completely oblivious to a huge discussion happening around it. An app built on top of Friendfeed, Noiseriver, is trying to fix this. But Friendfeed is just a microcosm of a data siloing problem that the entire web faces. We're seeing a huge movement for something called "data portability", but this seems to have nothing to do with comments and everything to do with profile information and friends lists.

This is important too, but not the same goal. Sure, it's great to meet up with your friends on a completely different site, but does nothing for helping you connect with complete strangers. The real key to connecting people digitally is unifying the conversation. The argument against this is wanting to talk exclusively with one's friends, but this is unfounded since the program could easily bubble friends' comments to the top and only display other comments upon request. So I'm asking for a solution to comment fragmentation. We need an open API for posting threaded comments across the web. Any site choosing to accept open comments should accept a POST request stating,

"I am commenting on this URL in reply to that comment, and here is the text or URL of my comment".

And then of course you could GET for the current set of comments. Now anyone or any app involved in the discussion around a URL will be on equal footing and have access to all that is being said about it, because the silo now has a series of tubes running through it. Perhaps we can't realistically expect arbitrary sites to handle this kind of load, so some kind of glue, such as Gnip, may need to get involved. But this is the solution, and it will kick ass when it is created and people start using it. There is so much that the internet makes possible, so there is no reason to believe that what we have now is as good as it gets.