Papers for Backend Developers

The Google File System
Scaling Memcache at Facebook
In Search of an Understandable Consensus Algorithm (Extended Version)
ZooKeeper: Wait-free coordination for Internet-scale systems
What’s Really New with NewSQL?
Kafka: A distributed messaging system for log processing
Dapper, a large-scale distributed systems tracing infrastructure
Making reliable distributed systems in the presence of software errors
Overload control for scaling Wechat microservices
Bitcoin: A Peer-to-Peer Electronic Cash System
Large-scale cluster management at Google with Borg
Don’t Settle for Eventual: Scalable Causal Consistency for Wide-Area Storage with COPS
Meltdown: Reading Kernel Memory from User Space
Efficient IO with io_uring

System Design

The Google File System (GFS) [1]

This is a landmark industry paper. Unlike academic approaches, industry solutions often address problems with limited resources, time, or personnel. GFS is an excellent paper because its authors deeply understand the problem and make thoughtful trade-offs. For instance, GFS opts for a single master design. While single masters are often criticized for scalability issues, this choice simplifies complexity and, with a well-thought-out design, meets the near needs of the system. However, as GFS needed to scale by several orders of magnitude, the single-master design eventually became a bottleneck. You can read more about this in a retrospective interview with a GFS engineer [2].

Scaling Memcache at Facebook [3]

This paper discusses how to scale caching systems at a large scale. It trades consistency for scalability to solve practical problems encountered in production. One key takeaway is: "Separating cache and persistent storage systems allows us to independently scale them." Separation is a fundamental principle in designing distributed systems and microservices. Related papers offer further reading: [4][5][6].

yfractal / blog