Red mark

Stories about development, systems and awesome music.

High Scalability – High Scalability – Tumblr Architecture – 15 Billion Page Views a Month and Harder to Scale than Twitter

, No Comments

With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do. Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers. One of the common patterns across successful startups is the perilous chasm crossing from startup to wildly successful startup. Finding people, evolving infrastructures, servicing old infrastructures, while handling huge month over month increases in traffic, all with only four engineers, means you have to make difficult choices about what to work on. This was[...]

Read More »

William Hertling’s Thoughtstream: Scaling A Web App 1,000x in 3 Days

, No Comments

When I’m not writing science fiction novels, I work on web apps for a largish company*. This week was pretty darn exciting: we learned on Monday afternoon that we needed to scale up to a peak volume of 10,000 simultaneous visitors by Thursday morning at 5:30am. This post will be about what we did, what we learned, and what worked. A little background: our stack is nginx, Ruby on Rails, and mysql. The web app is a custom travel guide. The user comes into our app and either imports a TripIt itinerary, or selects a destination and enters their travel details. They can expressed preferences in dozens of different categories (think about rating cuisine types of restaurants, categories of attractions, etc.). They can preview their travel guide live, in the browser, and order a high quality paperback version. Our site was in beta, and we’ve been getting a few hundred[...]

Read More »

iTerm2 – Mac OS Terminal Replacement

, No Comments

via iterm2.com If you’re anything like me, you’re probably dealing a lot with the terminal. I use it all the time because i’m working with Git and lots of branches, i create cron jobs and console applications, so i need a pretty good terminal. MacOS already has a great terminal but there are a lot of stuff missing from it, just because MacOS isn’t meant to be used like Linux. Anyhow, iTerm is a great terminal replacement and i highly recommend it

Read More »

Next-generation sharing economies: why real-time matters most — Tech News and Analysis

, No Comments

As it matures, the driving force of the sharing economy will become time, and the companies that can do business in real-time will occupy a more strategic, and profitable, place in the ecosystem. Fresh off its $1 billion valuation, Airbnb is the most common reference point for all manner of “this for that” pitches bouncing around the Valley right now, with many new ventures proposing to be the “the Airbnb of X.” But Airbnb is only one species of the sharing economy genus — a genus that will stratify over the next few quarters. Real-time makes your brand a hero Hotel Tonight is a great example of the flip side of the Airbnb coin. It focuses on real-time reservations, and the real-time use of latent capacity. Airbnb’s transactions typically take place five or more days in advance of a stay, and any requests inside that window are put on a standby[...]

Read More »

Redesigning the Country Selector – Baymard Institute

, No Comments

Using principles of progressive enhancement we turn a standard drop-down into an advanced auto-complete field. This means the drop-down remains accessible, while providing a much better experience in modern browsers – handling typos, multiple spelling sequences, synonyms and prioritization. You can read the more about the design process and usability aspects in our article on Smashing Magazine: Redesigning the Country Selector. via baymard.com Very interesting approach

Read More »

How GitHub Uses GitHub to Build GitHub

, No Comments

How GitHub Uses GitHub to Build GitHub September 21, 2011 / Helsinki, Finland / Frozen Rails 2011 Build features fast. Ship them. That’s what we try to do at GitHub. Our process is the anti-process: what’s the minimum overhead we can put up with to keep our code quality high, all while building features *as quickly as possible*? It’s not just features, either: faster development means happier developers. Slides References Merlin Mann’s “Mud Rooms, Red Letters, and Real Priorities” sinatra_auth_github Why GitHub Hacks on Side Projects How GitHub Works Scott Chacon’s “GitHub Flow” via zachholman.com The actual presentation is hosted in the source website. Please visit. It basically describes an awesome productive way to work around your company or build your product

Read More »

Bootstrapping a Software Product // Speaker Deck

, No Comments

The Speaker Garrett Dimon Garrett is the founder, designer, and developer of Sifter, a simple hosted bug and issue tracking application. View Speaker Details via speakerdeck.com Another great presentation on building a web application designed to make profit. I think the title says it all

Read More »

labs | CodeIgniter/ActiveRecord setup to use master + slave db replication

, No Comments

This is how you can set up CodeIgniter to direct mysql queries to different read/write hosts in your db replicated environment, using a db_slave for your SELECT’s, and a db_master for the INSERT/UPDATE/DELETE queries. File: application/config/database.php Specify the different database hosts in the database config file: < ?php if ( ! defined(‘BASEPATH’)) exit(‘No direct script access allowed’); $active_group = "master"; $active_record = TRUE; # db_master $db['master']['hostname'] = "host1"; $db['master']['username'] = "username"; $db['master']['password'] = "password"; $db['master']['database'] = "exampledb"; $db['master']['dbdriver'] = "mysql"; $db['master']['dbprefix'] = ""; $db['master']['pconnect'] = FALSE; $db['master']['db_debug'] = TRUE; $db['master']['cache_on'] = FALSE; $db['master']['cachedir'] = ""; $db['master']['char_set'] = "utf8"; $db['master']['dbcollat'] = "utf8_general_ci"; #db_slave $db['slave']['hostname'] = "host2"; $db['slave']['username'] = "username"; $db['slave']['password'] = "password"; $db['slave']['database'] = "exampledb"; $db['slave']['dbdriver'] = "mysql"; $db['slave']['dbprefix'] = ""; $db['slave']['pconnect'] = FALSE; $db['slave']['db_debug'] = TRUE; $db['slave']['cache_on'] = FALSE; $db['slave']['cachedir'] = ""; $db['slave']['char_set'] = "utf8"; $db['slave']['dbcollat'] = "utf8_general_ci"; … File: application/core/My_Model.php Add this into My_Model: < ?php class MY_Model extends[...]

Read More »

High Scalability – High Scalability – Must see: 5 Steps to Scaling MongoDB (Or Any DB) in 8 Minutes

, No Comments

Jared Rosoff concisely, effectively, entertainingly, and convincingly gives an 8 minute MongoDB tutorial on scaling MongoDB at Scale Out Camp. The ideas aren’t just limited to MongoDB, they work for most any database: Optimize your queries; Know your working set size; Tune your file system; Choose the right disks; Shard. Here’s an explanation of all 5 strategies: Optimize your queries. Computer science works. Complexity analysis works. A btree search is faster than a table scan. So analyze your queries. Use explain to see what your query is doing. If it is saying it’s using a cursor then it’s doing a table scan. That’s slow. Look at the number of documents it looks at to satisfy a query. Look at how long it takes. Fix: add indexes. It doesn’t matter if you are running on 1 or 100 servers. Know your working set size. Sticking memcache in front of your database is silly.[...]

Read More »