techdiagrams.net
← All posts

Hot shards and hot keys: when perfect hashing can't save you

Suchait Gaurav· · 3 min read· — views distributed-systemsshardingscalabilitydatabases

Your sharding is perfect. Sixteen shards, consistent hashing, keys spread beautifully evenly. Then one post goes viral — and shard 3 is at 100% CPU while the other fifteen idle at 2%. Adding shards does nothing. Welcome to the hot-shard problem, the failure mode that perfect hashing cannot save you from:

One viral key routes 95% of traffic to shard 3 while the hash ring stays perfectly balanced

A real techdiagrams document — download the JSON and import it into the editor via File → Import.

Why "just add shards" fails

A hash function distributes keys evenly — it says nothing about the traffic per key. Real workloads are zipfian: a handful of keys carry a huge share of all requests. The celebrity's post, the viral video, the flash-sale SKU, the one tenant who is 40% of your revenue. A perfect hash dutifully places that key on exactly one shard — and now 30% of your entire load has a single machine's ceiling. Re-hashing, re-sharding, doubling the fleet: the hot key still lands somewhere, whole.

There's a quieter cousin: the hot shard without a hot key — range-sharded data where the newest range takes all the writes (timestamp-prefixed keys, auto-increment IDs). Different cause, same melted machine, and worth ruling out first: is it one key, or one range?

The mitigation ladder

Work down this list; each rung is more invasive than the last.

0 · Detect it. You can't fix what you can't name. Per-shard CPU tells you which shard; only per-key metrics tell you which key. A top-K heavy-hitters sketch (count-min + a small heap) is cheap enough to run always-on — and turns "shard 3 is sad" into "post 8412 is why."

1 · Cache the hot key — kills read heat. The viral post is the same bytes for everyone; serving it from a near-cache (or the client) removes 99% of reads before they touch the ring. Two traps hide here: stampede — when the cache entry expires, ten thousand requests miss simultaneously and melt the shard you just saved (fix: request coalescing — one loader, everyone else waits — plus TTL jitter); and uselessness against writes — likes, counters and bids don't cache.

2 · Salt the key — kills write heat. Split the hot key into key#0 … key#15, each hashing to a different shard; writers pick a salt at random, readers fan in and merge. This is the classic sharded counter: sixteen partial counts, summed on read. You've traded one hot writer path for a slightly costlier read — almost always the right trade. The subtlety: salt only the hot keys (detection, again), because salting everything multiplies every read by N.

3 · Rebalance with a bound. Bounded-load consistent hashing caps any shard at c× the average load and spills the excess to the next node on the ring — it contains moderate skew automatically. For range shards, split the hot range and move half — this is exactly what DynamoDB's adaptive capacity and Bigtable's tablet splitting do for you. Know the ceiling: no rebalancer can split a single key. That's rungs 1 and 2's job.

4 · Isolate the whale. When one tenant or one entity is persistently enormous, stop treating them as ordinary data: give them a dedicated shard (or a dedicated cell), with their own quotas and their own blast radius. The heavy-hitter stops being an incident and becomes a line item. This is the pattern behind "celebrity infrastructure" at large social networks — the biggest accounts genuinely run on different plumbing.

The anti-patterns that manufacture hot shards

Worth naming, because they're all avoidable at design time: timestamp-prefixed keys (all writes hit the newest range — prefix with a hash instead), auto-increment partition keys (same disease), user_id as the partition key for a B2B product (your largest customer is a hot shard), and caching without stampede protection (rung 1 done badly is worse than not doing it).

See it melt, then fix it

The flash-sale template in our gallery composes the full defense — sharded counters, token buckets, cache warming — as an importable architecture. Or import this post's diagram, put real numbers on the edges, and watch what 95%-on-one-key does to a design before your traffic does it for you.

Get the next post in your inbox

Engineering notes from building an architecture editor that runs — React traps, simulation internals, AI product patterns. No spam, unsubscribe anytime.