What is IP rate limiting?

IP rate limiting is a mechanism that caps the number of requests an IP address can send to a server within a given time period. If you exceed the limit, the server temporarily refuses your requests, typically with an HTTP 429 error ("Too Many Requests"). It is one of the most common tools for protecting servers against abuse, denial-of-service attacks and automated scraping.

You have probably encountered it without realising. When a site shows "Too many attempts, try again in 5 minutes" after several failed logins, that is rate limiting. When an API returns an error after a certain number of calls per minute, that is rate limiting. When a search engine temporarily blocks your queries because you searched too fast, that is rate limiting too.

Why it is necessary

A web server has finite resources: CPU, memory, bandwidth, database connections. Every request consumes a share of those resources. Without any limitation, a single user (or an automated script) could monopolise the server by sending thousands of requests per second, making the service inaccessible for everyone else.

Rate limiting protects against several types of threats:

DDoS and brute force attacks. A distributed denial-of-service attack (DDoS) floods a server with requests to make it unavailable. Rate limiting alone is not enough to block a massive DDoS attack (which comes from thousands of different IP addresses), but it limits the damage any single IP can cause. For brute force attacks (trying thousands of passwords), rate limiting is the first line of defence: capping login attempts at 5 or 10 per minute makes the attack impractical.

Abusive scraping and crawling. Bots that extract site content (prices, articles, data) send hundreds of requests per second. Rate limiting slows these bots to the point where mass extraction becomes economically unviable (too slow, too much server time required).

API abuse. Public APIs (weather, geolocation, AI, payments) have a per-request cost for the provider. Without rate limiting, a user could consume disproportionate resources, whether by mistake (a loop in their code) or intentionally. Most APIs impose limits per API key and per IP address.

Fairness between users. Even without malicious intent, a "greedy" user (refreshing a page 50 times per minute, or a poorly written script) can degrade the experience for others. Rate limiting ensures a fair share of resources.

How it works technically

The principle is simple: the server associates each request with an IP address, counts the number of requests received from that IP within a time window, and rejects requests that exceed the defined threshold. But the concrete implementation varies depending on the algorithm used.

Fixed Window. Time is divided into fixed intervals (for example, one-minute windows). The server counts requests per IP within each window. If you hit 100 requests in the current window, subsequent ones are refused until the next window begins. It is the simplest algorithm, but it has a flaw: if you send 100 requests at the end of one window and 100 at the start of the next, you have sent 200 requests within seconds, double the intended limit.

Sliding Window. Instead of fixed windows, the server looks at the last N seconds at any given moment. Each request is counted over a sliding period. This smooths the distribution and eliminates the fixed window problem, but requires more memory and computation on the server side.

Token Bucket. Each IP has a "bucket" containing a fixed number of tokens. Each request consumes a token. Tokens regenerate at a constant rate (for example, 10 per second). If the bucket is empty, the request is refused. This model allows short bursts: you can send many requests at once if your bucket is full, then you must wait for regeneration. It is the most widely used algorithm in production because it is both flexible and predictable.

Leaky Bucket. Requests enter a queue (the bucket) and are processed at a constant rate (the leak rate). If the bucket is full, new requests are rejected. Unlike the Token Bucket, the Leaky Bucket enforces a strictly constant output rate, with no bursts. It is useful when you want a perfectly even processing flow.

Where rate limiting is applied

Rate limiting can be applied at different levels of the network infrastructure, and in practice it often is: multiple layers are stacked on top of each other.

At the web server level. Nginx, Apache and most web servers include rate limiting modules. Nginx, for example, uses the Leaky Bucket algorithm with its limit_req directive. You define a number of requests per second per IP, and Nginx handles the rest. It is effective for basic protection, but the server still has to receive and partially process each request before rejecting it.

At the reverse proxy or load balancer level. Placing rate limiting in front of the application servers (on a reverse proxy like Nginx or HAProxy, or a load balancer) allows requests to be blocked before they reach the application. This is more resource-efficient.

At the CDN or web application firewall (WAF) level. Services like Cloudflare, AWS WAF or Akamai apply rate limiting at the network edge, before the request even reaches your infrastructure. This is the most effective layer against volumetric attacks, because the request is blocked as close to its source as possible.

At the application level. The application itself can implement rate limiting, often with more granularity: per logged-in user, per endpoint, per operation type. For example, an API might allow 1,000 GET requests per minute but only 10 POST requests on an account creation endpoint, because abusive account creation attempts are a common attack vector.

HTTP headers related to rate limiting

When a server applies rate limiting, it typically communicates limit information via HTTP headers in the response. There is no universal standard, but the most common conventions are:

X-RateLimit-Limit: the maximum number of requests allowed in the current time window. X-RateLimit-Remaining: the number of requests remaining before hitting the limit. X-RateLimit-Reset: the time (as a Unix timestamp or remaining seconds) when the window resets and the counter restarts. Retry-After: a standard HTTP header indicating how long to wait before retrying, returned with the 429 response.

An IETF draft (RFC 6585 for the 429 status code and more recent proposals) attempts to standardise these headers, but in practice each API uses its own convention. If you are building an application that consumes APIs, always read the API documentation to understand how it communicates its limits.

Why IP alone is not always enough

IP-based rate limiting has a fundamental problem: an IP address does not always correspond to a single user.

CGNAT. Many mobile carriers and some ISPs use CGNAT (Carrier-Grade NAT), which makes hundreds or even thousands of users share a single public IPv4 address. If you limit an IP to 100 requests per minute, you are actually limiting all users behind that CGNAT to a combined total of 100 requests. A single normal user can exhaust the quota for everyone else.

Corporate and university networks. Thousands of employees or students may access the internet through a single public IP address (or a small number of addresses). IP-based rate limiting penalises them collectively.

VPNs and proxies. VPN users often share an IP address with hundreds of other people. Strict IP-based rate limiting can block legitimate users simply because they are using the same VPN server as a bot.

Dynamic IP addresses. Conversely, an attacker can easily change IP addresses (using a proxy pool, a botnet, or simply toggling airplane mode on their phone) and bypass IP-based rate limiting. If the limit is per IP and the attacker has access to 10,000 addresses, the effective limit is multiplied by 10,000.

This is why modern rate limiting systems generally combine IP with other identifiers: API key, authentication token, browser fingerprint, session cookie. IP remains a useful criterion, but it is rarely sufficient on its own.

What happens when you are rate limited

The most common response is an HTTP 429 status code (Too Many Requests). The server may also return a 503 (Service Unavailable) if the limitation is due to general overload rather than an individual threshold being exceeded.

Some services do not completely block access but degrade the service instead: they slow down responses (throttling), reduce the quality of results, or add a CAPTCHA to verify you are human.

Other services block silently without returning an explicit error. They continue to respond with a 200 (OK) status code but return empty data or degraded results. This "silent rate limiting" (shadow banning) is common on social networks and some search engines, and it is designed to be difficult for bots to detect.

How developers handle rate limiting

If you are building an application that consumes an API, you need to handle rate limiting properly. The standard method is "exponential backoff": when you receive a 429 response, you wait a given time before retrying, and you double that time with each successive failure. For example: first retry after 1 second, second after 2 seconds, third after 4 seconds, and so on.

Best practices include:

Reading the Retry-After and X-RateLimit-Reset headers to know exactly when to retry, instead of guessing. Implementing a cache to avoid repeating the same requests. Batching requests when the API allows it. Spreading requests over time rather than sending them in bursts. Setting up a circuit breaker that stops calling the API for a period if 429 errors accumulate, to avoid making the situation worse.

Rate limiting and privacy

IP-based rate limiting means the server identifies and tracks requests by IP address. This is a form of personal data processing (the IP address being considered personal data in many jurisdictions, notably under the GDPR in Europe). Companies implementing rate limiting should in theory mention this processing in their privacy policy.

From the user's perspective, IP-based rate limiting also means your activity is being counted. If you use a VPN or proxy for privacy reasons, rate limiting can paradoxically penalise you: you share an IP with other users and the shared quota runs out faster.

Bypassing rate limiting

Changing your IP address (via a VPN, a rotating proxy, or airplane mode on mobile) allows you to bypass rate limiting that is based solely on IP. Commercial scraping services use pools of thousands of residential IP addresses to distribute requests and stay under per-IP thresholds.

But modern protection systems do not rely on IP alone. They combine browser fingerprint analysis, browsing behaviour (click rhythm, mouse movements, time spent on pages), CAPTCHAs, and request pattern analysis to distinguish humans from bots, even when the IP changes with every request.

The race between rate limiting systems and bypass techniques is ongoing. Methods that work today may be detected tomorrow. If you are on the server side, the best strategy is to combine multiple layers of protection rather than relying on IP alone.

Examples of common limits

These limits vary enormously depending on the service, pricing plan and context. A paid API generally offers much higher limits than a free plan. A sensitive endpoint (login, payment, account creation) has much lower limits than a data-reading endpoint.

IP rate limiting is an essential protection for any server exposed to the internet. It prevents abuse, protects resources and ensures fair access. But IP alone is an increasingly unreliable identifier with CGNAT, VPNs and proxies. Modern systems combine IP with other signals to distinguish legitimate traffic from abusive traffic, and it is that combination that makes the difference between effective protection and a simple filter that is easy to bypass.