Emanuele Micheletti

I told 14 API clients to retry in a day. Four went to sleep.

·

Disclosure: this post was written with the assistance of an LLM, which also wrote the test scripts, ran them on my machine and read the SDK sources with me. The question and the choice of what to test are mine. I have reviewed everything. Every number in a table was measured. The limits quoted from source code are marked as such. The scripts and the raw captures are linked at the end.

Retry-After is advisory. RFC 9110 says how long a client ought to wait, RFC 6585 says a 429 may carry it, and nothing obliges the client to follow it. Each SDK decides for itself, each language port decides again, and the decision changes between versions: on 10 November AWS changed the default retry behaviour of its SDKs.

I wanted the behaviour on the wire, side by side. I put a proxy in front of 14 API clients, answered every request with 429 Too Many Requests, and recorded when each request arrived. Then I added a Retry-After header and changed its value. Afterwards I read each client's retry code and checked the numbers against it.

For one call that always fails:

The header is applied unevenly, and where it is applied it is applied literally: the client's timeout does not cover the wait, and six of the seven clients use the value as the wait, with no jitter.

The Setup

The proxy is Proxelar, with a Lua script that answers every request itself. Nothing reaches the real APIs, so no account or key is needed.

local status = tonumber(os.getenv("STATUS") or "429")
local hdr, val = os.getenv("HDR") or "", os.getenv("VAL") or ""

function on_request(request)
  local headers = { ["Content-Type"] = "application/json" }
  if hdr ~= "" then headers[hdr] = val end
  return { status = status, headers = headers, body = error_body_for(request.url) }
end

That is the core of it. The full script also returns an error body in the shape each API uses. Each client makes one call with its default settings: a chat completion, a ListTables, a customer list. Proxelar exports a HAR file with the arrival time of every request, to the millisecond. The gap between two arrivals is the client's sleep plus its own processing and any connection setup. Where a client waits exactly what it is told, the gap exceeds the value by 2 to 14 ms, which bounds that overhead.

The 14 clients are 8 products, all at the latest version on 18 November 2026:

For AWS I also ran the versions of the day before the change: boto3 1.43.127, and the JavaScript client 3.1165.0 installed with npm install --before=2026-11-10. The date matters, because in JavaScript the new behaviour ships in a shared package, @smithy/core 3.40.0, and not in the client.

The proxy is Proxelar 0.7.0, the Python clients run on Python 3.13.18 and the Node clients on Node 26.8.1. The OpenAI and Anthropic Node SDKs and Octokit spoke HTTP/2 to the proxy, the others HTTP/1.1. All clients ran at the same time through one proxy, on one laptop. In total: 11 scenarios, 304 calls, 1,156 requests.

I ran all of it twice. The first run used Proxelar 0.6.1, and Python 3.13 rejected its root certificate with Missing Subject Key Identifier, so the Python clients ran on 3.12. That was a defect in Proxelar, fixed in 0.7.0. The numbers below are from the second run, on 0.7.0 and Python 3.13. The first run also had the old JavaScript client installed with the new @smithy/core, so it measured the new behaviour twice. Leaving that client and Google Cloud Storage aside, whose request count is random, each of the other 266 calls sent the same number of requests in both runs.

What my 429 is not. It carries a status, a body and at most one header. Real APIs steer their own SDKs with more: GitHub sends x-ratelimit-remaining and x-ratelimit-reset, Stripe sends Stripe-Should-Retry, the OpenAI and Anthropic SDKs read an x-should-retry header, AWS has x-amz-retry-after. So this measures what each client does with the standard header alone. For Octokit, PyGithub and Stripe that is not what happens against the real service, and I say where it matters.

Retry-After, in Seconds

The table shows the first wait each client made after a 429. A dash means it did not retry. "Asleep" means it had sent one request and nothing else when I stopped it after 180 seconds.

Client02206186400
OpenAI Python0.40–0.46 s2.0 s20.0 s61.0 s—
OpenAI Nodeno wait2.0 s20.0 s0.42 s0.43 s
Anthropic Python0.42–0.49 s2.0 s20.0 s61.0 sasleep
Anthropic Node0.38–0.41 s2.0 s20.0 s61.0 sasleep
Google GenAI Python—————
Google GenAI Node—————
Google Cloud Storage0.78–0.82 s0.56 s0.25 s0.92 s0.52 s
boto30.64–0.72 s0.16–0.99 s0.92 s0.31 s0.43 s
AWS SDK for JavaScript0.67–0.84 s2.0 s5.3 s5.6 s5.8 s
Stripe Python—————
Stripe Node—————
PyGithubno wait2.0 s20.0 s61.0 sasleep
Octokit1.0 s1.0 s1.0 s1.0 s1.0 s
requests + urllib3 Retry0.00 s2.0 s20.0 s61.0 sasleep

The 0 and 2 columns are two runs each. The others are one run.

Two seconds. Seven clients wait 2.0 s: the OpenAI and Anthropic SDKs, the AWS SDK for JavaScript, PyGithub and the urllib3 adapter. The rest fall into three groups:

The limits, from the source. How far a client follows the header is one line of code in each. These are read, not measured, and they match every cell above.

ClientObeys Retry-After up toAbove that
OpenAI Python120 sraises without retrying
OpenAI Node60 sfalls back to its own backoff
Anthropic Python4,294,967 s (49.7 days)waits 49.7 days
Anthropic Node2,147,483 s (24.9 days)falls back to its own backoff
urllib3 Retry, and PyGithub through it21,600 s (6 hours)waits 6 hours
AWS SDK for JavaScriptits own backoff plus 5 swaits its own backoff plus 5 s

The AWS SDK for JavaScript treats the header as a floor. It waits the larger of its own backoff and the header value, and never more than its backoff plus 5 seconds. boto3 applies the same rule, but only to x-amz-retry-after.

61 seconds. The clients that obey here send 3 or 4 requests, so one call takes 122 seconds in OpenAI Python and in both Anthropic SDKs, and 183 seconds in the urllib3 adapter. PyGithub retries 10 times: it was on its fifth request at 244 seconds when I stopped it. With Retry-After: 20 it finished, after 11 requests and 200 seconds.

I then set timeout=5 on the OpenAI and Anthropic Python clients and repeated the 61-second case. The calls took 122.8 and 123.5 seconds. That timeout applies to each network operation of an attempt: connect, read, write. It does not cover the sleep between two attempts. Here it never fired, because the proxy answers at once.

One day. With Retry-After: 86400 four clients sent one request and went quiet for the 180 seconds I watched. From the limits above, they are not the same case. The two Anthropic SDKs would wait the full day, and then a second day for the second retry. The urllib3 adapter and PyGithub would wake after 6 hours, retry, and sleep another 6: 18 hours for the adapter's three retries, 60 hours for PyGithub's ten. OpenAI Python raised at once, because the value is above its 120 seconds. OpenAI Node retried after 0.43 s, because it is above its 60.

Zero. Retry-After: 0 is a legal value, and two clients then send every retry back to back. OpenAI Node takes the zero literally: 3 requests. PyGithub retries a 429 because the header is there, and has no backoff of its own: 11 requests. Through a local proxy that took 10 ms and 4 ms. Against a real server each retry would cost one round trip, and nothing more. OpenAI Python, both Anthropic SDKs and urllib3 treat 0 as absent and use their normal backoff.

Requests sent after a 429 with Retry-After: 0

No jitter on a Retry-After wait. Six of the seven clients use the header value as the wait, unchanged: the OpenAI and Anthropic SDKs, PyGithub and the urllib3 adapter. Asked for 2 seconds they waited between 2.002 and 2.009 s, on every retry of every run. Asked for 61, between 61.003 and 61.014 s. The same OpenAI and Anthropic SDKs spread their own first retry between 0.38 and 0.50 s. The AWS SDK for JavaScript is the exception: once its backoff grows past the header value the wait is jittered again, and its third wait at 2 seconds was 2.4 and 3.3 s.

A number of seconds keeps whatever spread the clients already had, because each one waits that long after its own response. The clients line up when the server sends each of them the time left until the same reset, or an HTTP-date. Then they all wake in the same second, and these six add nothing to it.

Other formats. Every client that reads the header in seconds also reads it as an HTTP-date, which has a resolution of one second and depends on the client's clock. OpenAI and Anthropic also honour retry-after-ms: 1500, a header of their own, and waited 1.5 s. A value that is neither a number nor a date, Retry-After: soon, is ignored by most. urllib3 raises InvalidHeader: Invalid Retry-After header: soon and does not retry, and PyGithub inherits that. A malformed header turns a retryable error into a different exception.

429, No Hint

Without the header, each client is on its own schedule. Each row of dots is one run, and each dot is a request.

Every request each client sent after a 429 with no Retry-After header, five runs each

One call produces between 1 and 12 requests, and lasts between a millisecond and two minutes.

ClientRequestsFirst to lastFirst wait, 5 runs
OpenAI Python31.3–1.4 s0.45–0.50 s
OpenAI Node31.3–1.4 s0.45–0.49 s
Anthropic Python31.2–1.4 s0.42–0.49 s
Anthropic Node31.2–1.4 s0.39–0.41 s
Google GenAI Python1——
Google GenAI Node1——
Google Cloud Storage9–1289.3–114.9 s0.12–0.75 s
boto340.9–5.4 s0.05–0.98 s
AWS SDK for JavaScript43.4–4.6 s0.01–0.99 s
Stripe Python1——
Stripe Node1——
PyGithub1——
Octokit414.0 s1.0 s
requests + urllib3 Retry43.0 s0.00 s

On a 503, three clients change:

My 503 carried the same body as the 429, so it is a throttling error with another status, not a real outage. The AWS SDKs classify the error by the exception in the body, which is why they treat the two alike.

AWS, Before and After 10 November

AWS SDK requests after a 429, before and after the 10 November 2026 default change

This is boto3 1.43.127, released the day before the change, calling DynamoDB:

14:02:48.731 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +0.000s
14:02:48.786 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +0.055s
14:02:48.889 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +0.158s
14:02:49.092 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +0.361s
14:02:49.497 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +0.766s
14:02:50.303 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +1.572s
14:02:51.909 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +3.178s
14:02:55.115 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +6.384s
14:03:01.520 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +12.789s
14:03:14.324 POST 429 https://dynamodb.us-east-1.amazonaws.com/  +25.593s

Ten requests, each wait double the previous one, starting at 50 ms. Across the five runs each delay differs by at most 6 ms. That is the old mode's setting for DynamoDB: 10 attempts, exponential from 0.05 s, no random factor. For other services the same old mode used 5 attempts with a random base.

boto3 1.44.5 sends 4 requests in 0.9 to 5.4 seconds, and no two runs look alike. For this call the change removed 6 requests out of 10 and added jitter.

The result is narrower than the chart suggests:

Stacking Retries

OpenAI's rate limit guide suggests wrapping the call with tenacity:

@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))
def completion_with_backoff(**kwargs):
    return client.chat.completions.create(**kwargs)

The client in that snippet has default settings, so it already makes 3 requests per call.

Requests sent for one logical call as retry layers are stacked

What makes the callRequestsTime to give up
OpenAI SDK alone31.3–1.5 s
LangChain ChatOpenAI31.8–2.8 s
LangChain ChatOpenAI().with_retry()97.8–8.9 s
SDK with the guide's decorator1823.2–27.0 s

Three runs each, same count every time, and the counts are the products: 6 × 3 and 3 × 3. For a call that never succeeds, the guide's own snippet sends 18 requests. The times include starting the Python process.

In Practice

A request timeout does not bound a call. On the OpenAI and Anthropic Python clients the timeout applies to network operations, and the sleep between attempts is outside it. The other clients that follow the header sleep the same way in their source. What can cut that sleep short differs. The synchronous Python clients call time.sleep, which nothing outside the thread interrupts. The async ones use a cancellable sleep. Both Node SDKs wake on an AbortSignal. So a deadline has to go around the SDK call in a form that client honours, or the wait has to be capped: max_retries=0 and a loop of your own, or retry_after_max in urllib3.

One retry layer. The OpenAI SDK already retries, and so do most of the others. A decorator or a .with_retry() on top multiplies the requests: 3 became 9 and 18. Either the wrapper or the SDK's own retries have to be set to zero.

A version and an entry point, not a product. OpenAI Python and Node disagree at 0, 61 and 86400. boto3 reads only x-amz-retry-after, and the AWS SDK for JavaScript reads both headers. Stripe Python retries a 503 or not depending on which of its two APIs makes the call. The AWS change itself arrived in a shared package, so the client's version number does not say which behaviour is installed.

For whoever sends the header. With the standard header alone, it reaches half of these clients. A value above 60 seconds is dropped by OpenAI Node, and a value above 120 makes OpenAI Python stop retrying. Zero makes two clients retry with no wait. Six of the seven use the value as it is, so a server that sends everyone the time to the same reset gets everyone back together.

What This Does Not Show

Files

run.py starts Proxelar, runs every client and exports the HAR. It expects the SDKs in ./py-main and ./node-main, and the pre-change ones in ./py-awsold and ./node-awsold. One scenario, one run, a 180-second window:

python run.py one-day 429 Retry-After 86400 1 180