Distributed Key-Value Store
Welcome to the distributed key-value store challenge!
You’ll build a distributed key-value store from scratch using Raft consensus, the same algorithm that powers etcd and Consul. If you haven’t already, read how it works before diving in.
Stages
Section titled “Stages”Build an in-memory key-value store exposed over a REST API.
Make your key-value store survive clean shutdowns by saving state to disk and restoring it when the server restarts.
Implement a write-ahead log so your key-value store survives unexpected crashes, not just clean shutdowns.
Implement Raft leader election so your cluster reliably elects and maintains a single leader, even under network partitions.
Replicate writes across your Raft cluster so every committed operation survives leader failures and is consistent across all nodes.
Prevent unbounded log growth by snapshotting your key-value store’s state and compacting the Raft log.
Dynamically add and remove nodes from a live Raft cluster using single-server membership changes, keeping the cluster online throughout.
Implement Raft joint consensus to safely resize a cluster by multiple nodes at once using overlapping C_old and C_new configurations.
Resources
Section titled “Resources”- Designing Data-Intensive Applications by Martin Kleppmann
- Database Internals by Alex Petrov
Papers
Section titled “Papers”- The Raft Paper by Diego Ongaro & John Ousterhout
Videos
Section titled “Videos”- Distributed Systems lecture series by Martin Kleppmann