Jaikido Blog

We’re planning to take Jaiku offline for an infrastructure upgrade between 10:00 am - 12:00 pm Pacific Standard Time (6:00 pm and 8:00 pm GMT) on Friday, October 17. The reason for the break is we’ll be undergoing a hardware expansion in our data center.

This is good news, because the upgrade will improve reliability of our infrastructure and significantly increase the bandwidth we have at our disposal.

The downtime is expected to last between 30 minutes and 2 hours. We apologize for the inconvenience and will be doing our best to minimize the disruption.

Snuck a couple small features in over the last couple days that we thought we might point out:

First, pagination has changed a little, changing from a list of page numbers to just older/newer links. This change will make some minor speed improvements but may not be the final decision so you are invited to give feedback. Also, by popular request, pagination was added to the overview page.

Second, in the list of feeds related to your Jaiku stream you will notice some have become links. As we get new updates from feeds we’ve also begun recording the url that the feed points to. They are only updated when there is actually new content pulled from the site so you may want to update something in a feed if you notice a link is missing.

We see quite a few questions from people asking about why their feed hasn’t updated or is slow to update (we’re working on speeding it up) so we thought we’d give everyone a little heads up on how it is all working and how we are working to speed it up.

Step 1: Figure out what needs to be updated

The common case is that we will update anything more than 30 minutes old, we don’t have a per-feed setting for update frequency at the moment, though we’d like one.

Step 2: Queue ‘em up

We grab a certain number of the items that haven’t been updated the longest and add them to a queue for processing.

A couple services requested that we poll less frequently so we don’t query any feeds from their servers more than once every minute — if we run into one of them that has happening too fast it gets kicked back in the queue for a bit. Unfortunately, there are only 1440 minutes in a day so this obviously doesn’t scale, we are interested in finding ways to alleviate this so that users of these services can have a better experience on Jaiku.

Step 3: Process them

Download the streams, find out if there are updates, add them to the site. The tricky part is again related to scale, we can only process so many of these every minute and unfortunately (well, a bit fortunately for us) we have quite more than enough users who have feeds becoming due to be updated every minute than we can actually run through in a minute, this means it can take more than the desired 30 minutes to update the feed.

Conclusion

Solutions to the update frequency problems will be adding more servers and bandwidth to the mix, which we are actively doing, so it will become more timely as we try to keep up with new users and their feeds.

Hope that helped explain some things for people.

Yikes, our apologies for neglecting our blog for so long, we’ve been distracted with making things faster and stronger and adding new features…

… but enough about us, go check out all the cool stuff that the community has been making, we’ve got all kinds of projects with support for Jaiku now over at http://devku.org/projects.

To keep up to date on the developments on these projects and the state of the API and even feature request stuff you can join the devku channel on Jaiku.

We’ve got our first Jaiku newborn! Big congratulations to Jyri and Ulla-Maaria, have you chosen a screen name yet? ;)

We’re going to have some downtime for a couple hours at about 10:30 EET so we can pretty ourselves up for Jyri’s talk at ETel.

Update: You’ve all been active, our database update is taking a bit longer than expected, sorry for the delays we’re crunching away as fast as we can.

Doing some database work tonight starting at midnight EET (so 14:00ish Pacific), should take a couple hours, see you on the flipside.

We’re going to have an hour or so of scheduled downtime for maintenance tomorrow morning, 7am to 8am CET (GMT+1), so feel free to sleep in.

Update: All went off without a hitch, good morning Europe :)