Backpressure and load shedding: the two answers to overload

Every system has a breaking point — a moment when incoming demand exceeds the capacity to serve it. What separates a resilient system from a fragile one is not avoiding that moment (you can't), but what it does in that moment. There are exactly two disciplined answers — backpressure and load shedding — and the systems that lack both share the same fate: a slow, total collapse that takes out everything, not just the excess. This is a deep look at both patterns: what they are, how they differ, and when each is the right tool.

The enemy both patterns fight: the unbounded queue

An unbounded queue absorbing 20k/s while draining at 2k/s — depth climbs without limit, latency explodes, memory runs out

A real techdiagrams document — download the JSON and import it into the editor.

Start here, because it explains why the other two patterns exist. The most natural reaction to a burst of traffic is to buffer it — put the overflow in a queue and work through it. It feels safe. It is a trap.

An unbounded queue doesn't absorb overload; it hides it, briefly, then amplifies it. Little's Law makes this precise: average latency = queue depth ÷ throughput. If arrivals exceed service rate, depth grows without limit, so latency grows without limit — and every request sitting in that queue is aging past the point anyone still cares about the answer. Then the queue exhausts memory and the process dies, taking the entire workload with it, including the requests that were perfectly serviceable.

This is the shape of most "the whole thing just fell over" outages. The fix is not a bigger queue. The fix is a bounded queue — because a bound forces a decision at the boundary: when we're full, what do we do? The two answers to that question are our two patterns.

Backpressure — push the "no" upstream

A fast producer, a bounded buffer, a slow consumer, and a 'slow down' signal flowing back to the producer when the buffer fills

Backpressure is flow control: when a downstream stage can't keep up, it signals upstream to slow down, and that signal propagates all the way back to the source. Nobody is allowed to produce faster than the slowest stage can consume. The overload never materializes because it's never generated.

The mechanism varies by substrate, but the principle is identical — a bounded buffer plus a way to say "wait":

Blocking / bounded channels. The producer calls queue.put() on a full bounded queue and blocks until space frees. Go channels, Java's ArrayBlockingQueue, a bounded asyncio.Queue — the block is the backpressure.
Pull-based / demand signaling. The consumer explicitly requests N items and the producer sends no more than that. This is the heart of Reactive Streams (request(n)), gRPC streaming flow control, and Kafka consumer poll loops — the consumer's appetite governs the producer's output.
TCP itself. The most battle-tested backpressure in the world: the receive window. A slow reader stops ACKing, the window shrinks, and the sender's write() blocks. Your OS has been doing backpressure under every socket you've ever opened.

The defining precondition: you control both ends. Backpressure is a conversation, and a conversation requires a cooperative party upstream who will actually slow down when told. That makes it the right pattern for everything inside your system — a data pipeline, a stream processor, a producer-consumer stage, a worker reading from a queue. Download this diagram.

Load shedding — when you can't say "slow down," say "no"

An admission controller at the edge dropping low-priority requests with 503 while admitting high-priority ones to a protected, healthy core

Now the case where backpressure is impossible. You cannot tell a million users' phones to slow down. You cannot pause the open internet. There is no cooperative upstream to signal — just an unbounded firehose of demand you don't control. Backpressure has nowhere to push.

So instead of "slow down," you say "go away" — deliberately, and immediately. Load shedding is admission control at the boundary: when you're at capacity, you reject the excess fast (an HTTP 503 with Retry-After) rather than accept it into a queue that will time out. The counter-intuitive truth at its core: serving 90% of requests well is far better than serving 100% of them terribly. A shed request fails in a millisecond and frees the resource; an accepted request in an overloaded system consumes CPU, memory and a connection slot only to time out anyway — dragging down the requests that could have succeeded.

Doing it well is about shedding the right things:

Priority-aware shedding. Drop the batch export before the checkout; drop the anonymous scraper before the paying customer; drop the "nice to have" before the "load-bearing." Shedding is only as good as your ability to rank requests.
Shed the doomed. A request that has already blown its deadline (its client gave up 200ms ago) is pure waste — process it and you burn capacity producing an answer no one is waiting for. Deadline propagation lets every stage drop work that's already dead on arrival.
Latency-based, not just count-based. Sophisticated shedders (CoDel-style) watch queue sojourn time rather than a fixed depth — if requests are sitting too long, start dropping, because rising latency is the real signal of overload.
Graceful degradation (brownout). A softer form: under load, shed features instead of requests — skip the personalization, serve a cached recommendation, drop the real-time pricing. The request succeeds, just with less. Download this diagram.

Which one? A single question decides

One decision: can you make the source slow down? Yes leads to backpressure; no leads to load shedding — and at the edge, backpressure becomes shedding

Both patterns answer the same question — demand exceeds capacity, now what? — and one property tells you which to reach for:

Can you make the source slow down?

Yes → backpressure. You control the producer (it's your own pipeline, your own stream, your own worker pool). Coordinate. Push the "no" upstream and let the whole chain settle to the pace of its slowest link.
No → load shedding. The source is the untrusted edge — thousands of clients, the public internet, anything that won't cooperate. Be unilateral. Drop the excess at the door.

And the elegant connection: backpressure degrades into load shedding at the boundary of your control. Inside your system, "slow down" propagates upstream from stage to stage. But at the outermost edge — where the upstream is a stranger's device — there's no one left to push back on, so the only honest expression of "slow down" is "I'm dropping this." Load shedding is what backpressure becomes when it runs out of cooperative upstream. Download this diagram.

They are not rivals — use both

A well-built system layers them: load shedding at the untrusted edge (the API gateway rejecting the flood) and backpressure through the internal stages (each service pacing the next). Together they replace the seductive, fatal unbounded queue with a system that always gives an honest, bounded answer — either "wait" to those who'll listen, or "no, retry later" to those who won't.

And critically, both are what stop an overload from becoming a cascade. Without them, an overwhelmed service slows, its callers pile on retries, the retries multiply the load, and the failure spreads outward — the retry storm we've covered, where a small overload becomes a total outage. Backpressure and load shedding are the circuit breakers on that chain reaction: they convert "everything collapses" into "the excess is refused, the core survives."

The mark of a mature system isn't that it never hits its limit. It's that when it does, it degrades on purpose instead of dying by accident. Import these diagrams into techdiagrams.net and find the boundaries in your own architecture — every place demand meets capacity is a place that needs one of these two answers, and the unbounded queue is never the third.