Rate Limiting

In technical terms

Algorithm families: fixed window (cheapest; doubles up at window boundaries), sliding window (accurate, more state), token bucket (allows bounded bursts), leaky bucket (strict pacing). Distributed versions choose between a shared store (Redis), per-node counters with async roll-up, or enforcement at the gateway.

Why it appears in interviews

It is the smallest component that still forces every senior habit in one answer: clarifying the rule (per-user vs global vs per-tenant), estimating state size, placing the counter for latency, and choosing what breaks when the counter store dies.

The common misconception

The algorithm is not the hard part. Placement and failure mode are. A limiter that needs synchronous cross-region consensus adds latency to every request; one whose Redis dependency fell over can take down the API behind it.

Trade-offs & when it hurts

Shared store gives precision at the cost of latency and availability; local + roll-up accepts seconds of over-admission for speed and resilience. Fail-open passes abuse through during outages; fail-closed turns the outage into a 429 storm. A product decision that must be owned, not inherited.

How to show it in an interview

State the rule before the algorithm: “Per-user, burst 10x sustained rate, and a 429 with Retry-After; token bucket per region, counters roll up to a shared store every 30 seconds, accepting brief over-admission rather than cross-region latency on every request.” The rule sentence and the admitted failure are the two parts panels score.

Questions this concept earns

• Design rate limiting for a public API with three pricing tiers and burst allowances.
• Which limits live per node, and which must be global: and what does that cost?
• The counter store dies at peak: do your users see 429s or do you fail open? Defend the choice.

Related