ETech: Final Notes to Myself

The theme for this year's ETech was "The Attention Economy." The amount of information available to people on the Internet today can be completely overwhelming. We need to divide our attention economically so that everything we need to (or want to) know about can be processed. A number of projects, products, and tools were presented to assist someone who is trying to deal with today's info-glut. There were also a number of talks discussing how we tend to focus or divide our attention, and what we might do better. Below are some notes to myself about what I saw, so that when I inevitably do forget, I'll have a place to look up this stuff.

Monday:
I attended a tutorial called "Scaling Fast and Cheap - How We Built Flickr" given by Cal Henderson. Flickr started out as part of an online game to be called "Game Neverending" (GNE), being developed by Ludicorp in Vancouver. It was originally implemented in Flash, which prevented it from scaling very far. I tried out the Flash version at ETech 04 and found it to be interesting but a bit slow. The current implementation seems fast enough to me and obviously scales, as there are over 2 million users and over 100 million pictures in Flickr today.

Cal covered quite a bit of ground over the course of the day.... Flickr was originally implemented using commodity (white box) hardware. They originally tried customizing the kernel and various other bits of software but found that the extra work involved was not justified by the small performance gains achieved. Cal had a great quote from Donald Knuth: "We should forget about small efficiencies, about 97% of the time." Here's a very small sample of the stuff we learned in the tutorial, some of which seemed like common sense but much of which could only be learned through experience:

Make sure you have enough rack space. Make sure you have enough power and networking in each rack. When you buy spares, ensure that you get identical spares for things such as disks, and not just equivalent sizes. This can be crucial when you need to replace a disk in a RAID set. If you must use a data center on the 24th floor, make sure that any equipment you buy (such as additional UPS systems to increase capacity) can actually fit in the elevator. Otherwise it might take a while to disassemble, move, and reassemble the equipment. Have as much redundancy as you can afford (hot spares, cold spares, spare disks, spare network gear, etc). 100M ethernet is probably fast enough for most applications. Software architecture is like a trifle (an English trifle, not a Canadian trifle). The 3 rules of enterprise system development: use source control; have a one step build; use a bug tracker. Filtering (X)HTML input in order to prevent cross-site-scripting attacks can be a royal pain. There are so many ways an attacker can format (eg random white-space or newlines between "java" and "script" in the word "javascript") or encode things (eg "j", j, j, j, %6A) that you need to be extremely clever if you're filtering with regular expressions.

If you want to know more, you'll need to buy Cal's book which is due out from O'Reilly, probably in the first half of 2006. I'm not sore what the title will be, but it will be authored by Cal Henderson and will be something about web applications.

Monday Evening:

For me, the highlight of the Monday evening keynotes was Bruce Sterling's "The Internet of Things." He packed alot of words and imagery into a fairly short talk. You can get a taste of what we heard in an available podcast. Unfortunately, a talk of this type would have been better earlier in the day. 8:30 PM was a bit late to get started.

Tuesday:

In the morning we were treated to a number of keynote talks. The 3 I found most interesting were Ray Ozzie's "Simple Bridge-building," Dick Hardt's "Who Is the Dick on My Site," and Linda Stone's "Attention: The 'Real' Aphrodisiac." Ozzie spoke about Microsoft's web "clipboard" technology, which looked very interesting. Hardt did one of his "Identity 2.0" talks, which are always entertaining. He shares a rapid-fire slide presentation style with Larry Lessig. Stone's talk was all the more interesting because a large percentage of the audience appeared to be paying more attention to their laptop screens than to the speaker. More information about all of these talks can be found by looking for ETech at Technorati.

Foldera Unveiled...

playsh...

Building More Useful Mashups...

Sigh....

This entry was started on March 15, 2006. It is now May 22, and I have unfortunately forgotten the rest of what I was going to write. I guess that if I'm going to be serious about blogging, I'll need to start setting aside time each week to do it.

Technorati Tags: ETech, ETech 06