Interview prep notes

Revision bc60fc3b5144b83369566fa45a52528733d7160a

distlock.md Raw

System Design Interview: Distributed Locks, Leases and Fencing Tokens

"How would you make sure only one worker processes this job?" is a common follow-up. A good answer covers leases and why a lock alone isn't enough.

The naive Redis lock

SET lock:job42 <random-token> NX PX 30000
  • NX only sets the key if it doesn't exist; PX gives it a 30 s expiry so a crashed holder doesn't block forever.
  • Release with a script that deletes the key only if the value is still your token, so you never delete someone else's lock.

Why expiry creates a problem

Suppose worker A gets the lock, then pauses (GC, network stall) for 40 s. The lock expires, worker B acquires it, and then A wakes up and writes anyway. Both think they hold the lock.

Fencing tokens

Have the lock service hand out a number that increases with every acquisition (A gets 33, B gets 34). Every write to the protected resource includes the token, and the resource rejects any token lower than the highest it has seen. A's late write with 33 is refused.

This requires the storage layer to check tokens, which is the point interviewers want you to notice: correctness comes from the resource, not just the lock.

Choosing a lock service

  • Single Redis: fast, fine for efficiency locks (avoiding duplicate work) where an occasional double run is harmless.
  • Redlock (majority of independent Redis nodes): debated; depends on timing assumptions.
  • ZooKeeper / etcd: consensus-based; use ephemeral nodes or leases tied to a session, and their revision numbers work as fencing tokens. Prefer these for correctness locks.
  • Database row lock or a conditional update on a version column is often enough and is the simplest answer.

Interview tip

Ask whether the lock is for efficiency or correctness before picking a design.


Disclosure: I work on Linkjob AI, real-time interview help.