Working on hashtags/users data… and a little setback

These days I’ve been working on getting the data needed to show the top tweeters/influencers for an hashtag.

For the less used hashtags that’s not easy, but for the more common ones I was already getting some interesting enough data and I was thinking about the possibility to publish a first beta this weekend.

Unfortunately, while working on this new feature, something got corrupted on Redis – the database where we collect our data – and had to go back to an older backup; you might already have noticed that the number of examined tweets displayed on hashtagify fell from over 50 to 43 millions.

Nothing serious, but this means that we’ll need to wait some more time for the first beta. I just hope to be able to publish something before I’ll leave for a 2 weeks vacation to Russia next week… but don’t hold your breath!

While working no this I also learned something interesting about how to execute batch jobs on node.js: I expected some complications from the non sequential nature of node, but I hadn’t forseen the full extent of the surprises! I’ll try to share this to thos interested in this kind of technical details an upcoming post.