Yesterday afternoon, during planned maintenance that was not intended to interrupt service, an issue arose that took down a critical database cluster. This brought down our entire network while our engineers worked feverishly to restore these databases and bring your blogs back online.
While you might feel like you’ve gotten used to seeing errors on Tumblr recently, know that this is absolutely unacceptable to our team, and unacceptable for a platform determined to be the best place in the world for your creative expression.
Frankly, keeping up with growth has presented more work than our small team was prepared for — with traffic now climbing more than 500M pageviews each month. But we are determined and focused on bringing our infrastructure well ahead of capacity as quickly as possible. We’ve nearly quadrupled our engineering team this month alone, and continue to distribute and enhance our architecture to be more resilient to failures like today’s.
We can’t apologize enough, nor can we thank you enough for putting up with these growing pains. We know how impossibly frustrating it is to see your work offline. But please always know that we truly care about your work as much as you do, and we have an incredibly capable team working incredibly hard to take good care of it.
Sorry we let you down today.
P.S. If our staff blog is ever unreachable, you can check our Twitter stream for updates.
P.P.S. Anyone that had a paid Directory Feature running through the last two days has had it automatically extended 36 hours.
This was fun! Top Gear came by our office last week and shot this segment about our team’s first cars.
Tumblr iPhone 2.0: Now available on the App Store
We’re excited about this one! The app has been recoded, rewired, and rethought from the ground up to offer you the very best Tumblr experience:
- Brand new interface: Browsing your dashboard, creating a post, managing multiple blogs — the most important features are within reach so you can interact with Tumblr effortlessly while you’re on-the-go.
- Creating a post: Now easier than ever! Sharing photos, videos, links, chats, text, quotes and audio is front and center. Advanced options like saving drafts and queuing posts are one swipe away.
- Messages: View and reply to messages for each of your blogs.
- Address book: Find people to follow from your phone’s address book.
- New users: Signing up is now native inside the app.
There are tons more features and details that are awesomely crafted.
A large segment of custom domain blogs experienced an outage earlier this afternoon with roughly 3.5 hours of intermittent availability.
The interruption stemmed from an issue in the way this blog traffic gets load balanced (distributed across our servers). This system has been undergoing a…
Twenty
A short film by William Wilkinson. (previously)
Tumblr Android 2.0.4: Now available on the Android Market
This update features:
- New, cleaner, better-looking post screens.
- Faster Dashboard loading — especially when switching screens.
- Reblogging to Queue.
- Lots of bugs squashed.
I’m still working with a few of you who are mysteriously unable to log in from certain devices. If you run into this issue, please email support and tell them Chris sent you.
As always, a big thanks to all the beta-testers and bug reporters. :)
