Summary: Most engineers answer the rate-limiting question with "store request counts in Redis." That is not wrong, it is the first sentence. The design work is everything after it: which algorithm handles bursts fairly, how to make the counter update atomic, what happens when one key gets hot, where in the topology enforcement belongs, and — the question that decides whether the limiter is a safety feature or an outage cause — what the system does when the limiter's own datastore dies.

The goal is not to stop users from calling the API. It is to ensure every user gets a fair share of platform resources while protecting the system from abuse. That framing decides most of the downstream trade-offs.

Why in-memory counters fail

The simplest design is a fixed window counter: allow 100 requests per minute, increment on each request, reject past the limit.

In a distributed system this silently breaks. Requests do not always reach the same server. With three instances each holding their own counter, a client making 100 requests to each has made 300, while every server still believes the limit is intact. The counters must live in shared state — most commonly Redis — so that every instance sees the same usage regardless of which one handled the request.

Algorithms, in order of sophistication

Fixed window has a boundary problem even once the state is shared. A client sending 100 requests at 12:00:59 and 100 more at 12:01:01 has sent 200 requests in two seconds, and both windows are technically within limit. The limit is correct and still unfair.

Sliding window removes the boundary effect by evaluating activity continuously over the trailing interval. The cost is storage and computation: an exact sliding window needs a timestamp per request, which at 100,000 requests/second is expensive.

Token bucket is what large systems generally prefer, because it handles bursts deliberately rather than accidentally. Each client owns a bucket with a capacity and a refill rate — say capacity 100, refill 10/second. Every request consumes a token; an empty bucket means rejection. A client that has been idle finds a full bucket and can serve a 100-request spike immediately, after which it is constrained to the refill rate. That is a far better experience than rejecting every burst outright, and it still enforces the long-run limit.

Redis, and why the update must be atomic

Redis contributes two things: shared state and atomic operations. The bucket is stored as userId, tokensRemaining, lastRefillTimestamp, and each request must calculate how many tokens should have accrued since the last refill, update the bucket, consume a token, then accept or reject.

Done as separate read-modify-write steps, this races: two concurrent requests can read the same tokensRemaining and both consume the same token. Wrapping the whole sequence in a Lua script makes Redis execute it atomically. This is the same class of concern as the Database Concepts Every Backend Developer Must Understand treatment of locks and isolation levels — concurrent access to shared state needs an explicit correctness mechanism, not hope.

The hot key problem

If a single API key produces 50,000 requests per second, every one of those requests updates the same Redis key. Redis becomes the bottleneck, and the rate limiter itself becomes the scaling constraint — precisely the failure mode it was installed to prevent.

Mitigations include sharding the key across multiple counters, local token caches on each instance that reconcile periodically, hierarchical rate limiting, and pushing enforcement to the gateway. Which one applies depends on the traffic shape; there is no default.

Where enforcement belongs

Not in every microservice. Enforcing at the edge — the API gateway — rejects abusive traffic before it consumes application compute, database connections or network bandwidth. Pushing the check inward means paying most of the request's cost before deciding not to serve it. This is consistent with how Designing a URL Shortener That Works Like Infrastructure treats the edge as the place where cheap decisions get made.

Multi-tenant limits

Plan tiers — Free at 100/min, Pro at 1,000/min, Enterprise unlimited — do not need different algorithms. They need different bucket parameters. The capacity and refill rate become configuration attached to the caller's plan; the token-bucket logic is identical for everyone. Keeping the policy in configuration rather than in code is the same instinct Encoding Team Standards applies to team rules.

Fail-open or fail-closed

When Redis goes down, the limiter has to choose:

Mode Behaviour Cost
Fail closed reject requests protects the platform, destroys availability
Fail open allow requests preserves availability, risks abuse

For customer-facing APIs most businesses prefer fail-open for a bounded period, on the reasoning that an outage caused by the rate limiter itself is usually worse than temporarily serving extra traffic. This is a business decision wearing engineering clothes, and it should be made deliberately rather than inherited from a library default. Dissecting the AWS Outage A Step-by-Step Breakdown of What Went Wrong is the cautionary case: control-plane dependencies that fail closed turn a component outage into a platform outage.

Multi-region

Running in US-East, Europe and Asia turns the limiter into a consistency problem. A globally consistent counter requires cross-region coordination on every request, adding latency to the hot path. Regional counters are fast but let a client exceed the nominal global limit by roughly the number of regions.

Most systems accept the slight inaccuracy in exchange for latency — the limit is a fairness mechanism, not an accounting ledger, and being approximately right at low latency beats being exactly right at high latency.

The design, summarized

  1. Enforce limits at the API gateway.
  2. Use Redis for shared distributed counters.
  3. Prefer token bucket for burst handling.
  4. Use Lua scripts for atomic updates.
  5. Support plan-specific bucket configurations.
  6. Monitor for hot keys.
  7. Decide fail-open versus fail-closed explicitly.
  8. Accept regional approximation in global deployments.

The algorithm is the small part. The real difficulty is making it work reliably across hundreds of servers, millions of users and multiple regions — fairness, distributed state, atomicity, bursty traffic, failure handling and global consistency, all at once.

Communicating the limit

A limiter that rejects silently trains clients to hammer harder. The API Design Patterns Nobody Teaches You makes the client-facing half of this case: return X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset on every response, and pair a 429 with Retry-After. The enforcement design in this article and the response contract in that one are two halves of the same feature.