Rate Limiting Guide
Rate limiting protects you from scrapers, brute-force bots, runaway retry loops and your own most enthusiastic customers — and with per-request LLM costs, it is now cost control too.
Three decisions define a limiter: the key (IP, user or API key), the algorithm (how bursts are treated), and the enforcement point (edge, gateway or application middleware).
Choosing an algorithm
Fixed windows are simple but allow double bursts at window boundaries. Sliding windows smooth that out for slightly more bookkeeping. Token buckets — a bucket refilling at a steady rate that each request drains — allow natural short bursts over a sustained rate, which is why they are the default in most production limiters. Leaky buckets instead smooth outflow to a constant pace, useful when a downstream dependency must never spike.
Where to enforce
Layer coarse to fine:
- Edge — Cloudflare rate-limiting rules stop volumetric abuse before it touches your origin
- Gateway — AWS API Gateway throttles and usage plans give per-key quotas for partner APIs
- Application —
@upstash/ratelimitbacked by Redis works from serverless and edge runtimes; express-rate-limit or rack-attack need a shared Redis store once you run more than one instance
Responding to limited clients
Return 429 with a Retry-After header and, ideally, the draft IETF RateLimit headers so well-behaved clients can pace themselves. Key authenticated traffic by user or API key rather than IP — corporate NATs and CGNAT put thousands of humans behind one address. Document your limits publicly and teach clients to retry with exponential backoff plus jitter.
Edge cases that bite
Give login, OTP and password-reset routes far stricter limits than the rest of the API. Weight expensive endpoints — one report export can cost a thousand health checks — with cost-based buckets. And decide failure behavior consciously: if the limiter's Redis is down, fail open for ordinary reads but fail closed on authentication endpoints.
Frequently asked questions
Should limits be per-IP or per-user?
Per-user or per-API-key wherever the request is authenticated; per-IP only for anonymous traffic, with generous thresholds because addresses are shared.
What limits should I start with?
Measure a week of real traffic, then set limits comfortably above your 99th-percentile legitimate usage and tighten from evidence, not guesses.
Does Cloudflare replace app-level limiting?
No — it kills floods cheaply at the edge, but per-user quotas, cost weighting and business rules still belong in your application layer.
What should a 429 response include?
A Retry-After header, rate-limit headers describing the window and remaining quota, and a problem+json body clients can log and branch on.