The batch APIs from OpenAI, Anthropic and Google cost 50% of their synchronous ones — same models, same output, half the bill. Almost nobody uses them, because “completes within 24 hours” is impossible to plan around, so teams pay full price by default. In reality most batch jobs finish in minutes. We measure that and tell you when the queue is fast enough to take the discount.
This is about large-language-model inference — the
/v1/batches endpoints and their equivalents. Not job
queues, not ETL, not batch processing in general.
Every major provider sells the same tokens twice: full price if you want the answer now, half price if you can wait. The discount is not small print — it is 50%, on every input and output token.
So why does almost nobody use it? Because the only promise you get is “within 24 hours”. No engineer can build a feature on that, and no product owner can plan around it. The rational move is to pay double and keep the latency you understand.
But 24 hours is a ceiling, not an estimate. Measured across real jobs, half finish in under three minutes and nine in ten inside twenty. The queue is usually fast. You just have never had a way to know when.
That is the entire product. We measure what the queue is actually doing right now, and answer one question: is it fast enough that this job should take the discount? When the answer is yes — and it usually is — you move that spend to half price without changing what your users experience.
Put your own numbers in.
Not every job can move. Interactive features, anything a user waits on, stays synchronous. But the batchable half — nightly enrichment, evals, backfills, classification, summarisation, report generation — is usually the larger half, and it is paying double today.
The batchable share is a slider because it is your number, not ours. We have not measured your workload and will not pretend to. The 50% discount is the providers’ published rate; everything else above is arithmetic on what you typed.
Coverage follows what people measure. Everything here is a real batch endpoint on a real provider — nothing is simulated.
| Provider | Endpoint | Discount | Their promise | Status |
|---|---|---|---|---|
| OpenAI | /v1/batches |
50% | 24 h | measured hourly |
| Anthropic | /v1/messages/batches |
50% | 24 h | prober written, not yet run |
| Google (Gemini) | batchGenerateContent |
50% | 24 h | prober written, not yet run |
| Mistral | /v1/batch/jobs |
50% | 24 h | accepted, not probed |
| Azure OpenAI | batch deployments | 50% | 24 h | accepted, not probed |
“Measured” means our own prober has submitted real jobs and timed them. “Accepted” means the API will store your measurements for that provider, but nobody has sent any yet. We would rather show you an empty column than imply coverage we do not have.
Any provider with a batch endpoint can be added — the schema is not OpenAI-shaped. What decides the order is where the measurements come from.
This is the chart the provider does not publish.
The distribution is not flat — it is front-loaded with a long, thin tail. That shape is why “up to 24 hours” is technically true and practically useless. The tail is real and it will bite you (one job in this dataset took eight hours), which is exactly why you want to be told when you are standing in it.
GET /v1/should-i-batch
?model=gpt-5.6-sol
&input_tokens=9720 ← you know these
&max_wait=15m ← your deadline
&risk=p90 ← how safe
← no output_tokens: you cannot know them
before the model has answered. We
estimate from measured jobs, or say
we cannot. Send max_tokens for a
ceiling, or output_tokens if you
genuinely have a figure.
{
"verdict": "batch_at",
"batch_at": "2026-08-26T02:00Z",
"your_limit_s": 900,
"batch_p90_s": 2460, ← misses it now
"batch_p90_at_target_s": 108,
"batch": { "p50_s": 372, "p90_s": 2460 },
"sync": { "p50_s": 41, "p90_s": 88 },
"saving_usd": 0.0647,
"required_patience_p90_s": 2372,
"n": 1180,
"confidence": "medium"
}
No deadline given? You get
required_patience_p90_s instead — the number you would
have to accept. Compare it to your own.
max_wait and we
answer against your limit — we never guess it.run_batch · run_sync ·
batch_at · insufficient_datainsufficient_data instead of a number. Every response
carries n.insufficient_data, never an exception. Your job still runs.GET /v1/wait?model=gpt-5.6-sol
{
"model": "gpt-5.6-sol",
"provider": "openai",
"p50_s": 372,
"p90_s": 2460,
"oldest_running_s": 28800,
"based_on": { "n": 14, "window": "1h" },
"vs_normal": 2.31,
"confidence": "medium",
"live": true,
"delayed_by_s": 0,
"measurement": "observed completions",
"not": "a prediction for your job"
}
No key needed. Without one you get the same shape with
"live": false and "delayed_by_s": 900.
The simplest thing we can offer. Start here.
Not every caller wants the decision logic. Sometimes you just need to know whether the queue is 40 seconds deep or 8 hours deep, and you will decide what that means yourself.
Read the last two fields. This is what jobs finishing right now actually took. It is not a forecast for the job you are about to submit — nobody can give you that honestly, and we have the data to prove it.
oldest_running_s is the one people miss. Completed jobs
describe the past. The oldest job still waiting is the earliest sign a
queue has stalled.
Try it without signing up. Anyone gets this endpoint at 15 minutes delayed, 60 calls an hour. That is enough to see the data is real and to link to it — and useless for routing, which is why we can give it away. Contributors get it live.
Same question, shifted in time. If the work isn't due until morning, the answer changes — and so does the price of waiting.
GET /v1/should-i-batch?at=2026-08-26T03:00Z answers for any
future hour. Schedule the job, don't sit and wait for it.
The client library reads four things off your batch job and sends them. It never touches the payload. This is the entire submission — there is no second, larger request.
{
"mode": "batch",
"provider": "openai",
"model": "gpt-5.6-sol",
"requests": 1,
"input_tokens": 9720,
"output_tokens": 4519,
"started_at": "…16:24:27Z",
"ended_at": "…00:24:12Z",
"status": "completed"
}
The client is open source. Read the 90 lines that do the sending before you install it — that is the point of publishing them.
That was the client library. This is the site you are reading, which is a separate question with a separate answer — and one we would rather state than leave you to infer from a cookie banner.
What we measure, in full: which page you are on, clicks on Get an API key, whether the spend calculator and the model selector were used, whether you scrolled as far as the API section, and whether a key was created. That list is the whole of it. If we add anything, the banner asks again.
Declining costs you nothing. Every number, chart and endpoint on this site behaves identically either way — there is no reduced version. Change your mind whenever you like: . Withdrawing deletes the cookies again.
If your browser sends Global Privacy Control, we take that as a no and never ask.
Every batch job you run is a measurement: when you submitted, when it landed, which model, how many tokens. Nobody publishes that — not the providers, not any monitoring service. It is too expensive to sample from the outside, because one data point costs one queue wait.
So the deal is simple. Send your measurements, get everyone else's. There is no other way this dataset can exist.
pip install batchwatch
with track("gpt-5.6-sol"):
client.batches.create(…)
if bw.should_batch():
…
No email, no password, no confirmation step. A key does not by itself carry any weight in the statistics — that is earned by measuring — so there is nothing to verify.
The label is your own note — “prod-pipeline”, “my laptop”. Nobody else sees it; it is how you tell your own keys apart when one has to be revoked.
Access is earned in recent measurements, not in signups.
| You | Dashboard | Submit API | Live API |
|---|---|---|---|
| Anyone, no account | full | — | 20 free calls, then /v1/wait, 15 min old |
| Account, no data yet | full | yes | 20 free calls, then /v1/wait, 15 min old |
| Contributing — 5+ measurements in the last 7 days | full | yes | 10,000/week, 5 min delayed |
| Paid | full | yes | live, no delay |
| Stopped contributing | full | yes | paused — any unspent free calls are still there |
Twenty free calls, no signup, no card. Point curl at it and see what it says about your model before you write a line of integration code. Asking you to instrument your pipeline first, on the promise that the answer might be worth it, is the wrong way round.
Calls that error don't count. The twenty are for finding out whether the
answer is useful, not for finding out how the query string is spelled.
Every response carries "trial": {"calls_left": 17}, so the
wall is never a surprise.
The dashboard is free forever, for everyone, no account. The API is what contribution buys, because the dataset is only possible because people contribute. Live figures are the one thing contribution does not buy — that is the paid plan, and it is what keeps the lights on for everyone else.
Five in seven days, not one a day. If you run batches twice a week you are still a contributor — we are asking you to stay current, not to change how you work.
Contributing buys the data. Money buys everything that is not data. You helped build the dataset, so you get to read it — at 5 minutes delayed, up to 10,000 calls a week. Live figures, higher volume, alerting, private per-account statistics, exports and a support commitment are what a paid plan is for. If you are hitting the limits, this product is almost certainly saving you more than the plan costs.
These terms will change. Saying otherwise before there is a business would be a promise made at the moment of least information, and we would rather not make one we might have to break. What we will commit to: 30 days’ notice before any change that reduces what contributors get, and we will not retroactively bill for data you have already read.
This is the summary. The full terms say the same thing at greater length. Nothing here is hidden in them.
We are stating this up front rather than burying it, because you are handing over data from infrastructure you may not personally own, and you should be able to justify that to whoever does.
Each model against its own 30-day baseline.
Median wait, 30 days. Weekends are cheap.
A backfill queued Friday evening behaves very differently from the same job on Tuesday morning. Nobody has been able to act on that before.
Duration vs input tokens, log scale both axes.
The obvious hypothesis — bigger jobs wait longer — does not hold. The spread is vertical, not diagonal. Your 40-token job waits the same as a 2-million-token one, which means splitting work to go faster is wasted effort.
The record no provider publishes.
| Start | Length | Model | × baseline | |
|---|---|---|---|---|
| 24 Aug 16:25 | 7h 58m | gpt-5.6-sol | 13.3× | measured |
| 18 Aug 13:10 | 2h 05m | gpt-5.6-sol | 4.1× | |
| 11 Aug 09:40 | 44m | claude-opus-5 | 3.4× | |
| 4 Aug 15:55 | 3h 20m | gpt-5.6-sol | 6.8× |
Through the 24 August outage, OpenAI's own status page read "fully operational" the entire time. measured
The honest limit of this dataset, stated up front.
19 jobs · same account · same model · same size 18 jobs median 3 min job 19 480 min
Perfect personal history predicted 3 minutes. The answer was 480 — wrong by a factor of 13. measured
So we never return "your job finishes at 14:32". We return a decision and the distribution behind it. A monitoring product that invents precision is worse than none.
How much data sits behind each answer. A model appears only with 20+ jobs from 3+ contributors — otherwise you could read one competitor's usage out of a thin bucket.
| Provider · model | Jobs, 30d | Contributors | Median | Status |
|---|---|---|---|---|
| openai · gpt-5.6-sol | 1,180 | 34 | 2m 41s | good |
| openai · gpt-5.6-terra | 602 | 21 | 2m 10s | good |
| openai · gpt-5.6-luna | 890 | 18 | 51s | good |
| anthropic · claude-opus-5 | 341 | 12 | 3m 05s | thin |
| google · gemini-3-pro | 47 | 4 | — | insufficient |
No account needed to contribute. That's deliberate.
POST /v1/calls
{ "mode": "batch",
"provider": "openai",
"model": "gpt-5.6-sol",
"requests": 1,
"input_tokens": 9720,
"started_at": "2026-08-24T16:24:27Z" }
→ 201 { "id": "c_7f3a…" }
PATCH /v1/calls/c_7f3a…
{ "status": "completed",
"output_tokens": 4519,
"ended_at": "2026-08-25T00:24:12Z" }
→ 200 { "duration_s": 28785,
"percentile": 99.4 }
The response tells you where your job landed in the distribution immediately. First value you get, and it costs us nothing.
Sync calls count too: mode: "sync".
GET /v1/wait how long is the queue?
GET /v1/should-i-batch should I use it at all?
GET /v1/conditions is it me or them?
GET /v1/distribution p50 … p99, filterable
GET /v1/outages historical incidents
GET /v1/coverage what we can answer
Every response carries:
"n": 1180
"confidence": { "level": "high", "why": … }
"plan_for_s": 4200
"freshness": { "last_measurement_age": "4 min" }
"trial": { "calls_left": 17 } while trialling
Sync measurements are what make should-i-batch possible.
Without both sides there is no trade-off to compute.