in Note

Diagrams, and the state of the union

Due to popular request, I’ve started retrospectively adding some diagrams to articles that really need them. First to get the treatment has been two-phase commit – by far the most popular article on this blog. The Dynamo article will be the next up, then 3PC and maybe the GFS and BigTable entries.

I’m using an old version of OmniGraffle, which came installed on my Powerbook G4. I recently replaced the power supply board in the G4 (which involved some hair-raising open Mac surgery) and am delighting in having all these great applications at my fingertips again. OmniGraffle makes diagrams for things like this so very easy. The effort expended in producing a diagram is far less than that for writing 1000 words, so if the old adage is true this is a very efficient way of producing content.

Although Real Work is consuming a lot of my time at the moment, I’ve been pretty good at finding time here and there to write for this blog. I’ve got two outward-facing goals here. The first is to make available some clear explanations for basic distributed systems theory and practice. I think there’s a niche for good work here – I don’t think any textbooks I have read adequately treat practice in a sufficiently theoretical way, and the theory textbooks can be too abstract to be accessible. Therefore I’ve been writing articles like the tour of FLP impossibility, the aforementioned introduction to two-phase and three-phase commit and the discussion of consensus in the context of lossy links. Continuing in this vein, I have plans to talk about Paxos, failure detectors, distributed spanner construction and some more simple, fundamental distributed algorithms such as leadership election.

My second ‘public-facing’ goal is to survey some of the more interesting (and occasionally less interesting) systems research, with a particular emphasis on real systems that exist and work. Hence the GFS, BigTable and PNUTS articles, and the recent series on OSDI papers. Part of my day job is being familiar with OSDI, NSDI, SOSP, HotOS, Mobi* etc. conferences and workshops, and by writing the articles I get the chance to consolidate my understanding, which is highly useful.

I’d be very interested to hear, by mail or by comment, if there are any particular topics that you’d like me to cover. I suspect the imminent article on Paxos will be popular (executive summary: it’s not that hard, especially if you already understand 3PC), but otherwise it’s hard to gauge what people are looking forward to reading on this blog. I even would enjoy picking up writing about algorithms that I abortively started to do. So help me out!