Around 68 seconds of serial work per row. Everything else about your timeline follows from how many of those you run at once — and from one platform limit most people meet the hard way.
Where the 68 seconds goes
A row is not one model call. With a repair loop it averages 2.25 attempts per completed row, at roughly 27 seconds per attempt.
That 2.25 is measured, not assumed, and it is the number people leave out of capacity planning. Only 12 of 80 rows passed on the first attempt in our production data. Planning against one call per row understates your wall-clock by more than half.
Concurrency is the only real knob
| Parallel workers | Rows per 5-minute window | 50,000 rows |
|---|---|---|
| 8 | ~30 | ~7 days |
| 16 | ~60 | ~3.5 days |
| 32 | ~120 | ~1.5 days |
Nothing else in the pipeline moves the number nearly as much. Prompt shortening, cheaper models, fewer validators — all marginal next to this.
If your generation is slow, the first question is not "which model is faster" but "how many requests am I actually running in parallel", and the answer is often a hardcoded constant somewhere.
The limit that silently loses rows
If you run generation on a request-triggered serverless platform, the request timeout caps your batch, and it is easy to get the arithmetic wrong in a way that loses work permanently.
Our setup: a 300-second request timeout, with a scheduler configured for a 600-second attempt deadline. The lower number wins. Batches sized against the scheduler's deadline get killed halfway through.
That would be merely wasteful, except for what happens to the in-flight rows.
Rows are locked to a processing state when a worker picks them up. With no
lease expiry, a killed batch leaves those rows locked forever — they never
return to the queue, and they are gone from your count without any error being
recorded.
Two fixes, and you want both:
- Size the batch against the real timeout, with margin. At 68 seconds per row and 300 seconds of wall clock, a serial batch fits about four rows; a batch of 8 concurrent workers fits about thirty.
- Add a lease expiry. Any row in
processingfor longer than a plausible maximum goes back topending. Without this, every crash costs you rows and you will not notice until the totals stop adding up.
Throughput does not have to mean concurrency inside one request
If your platform caps request duration, the cleaner architecture is many small scheduled invocations rather than one long one. A five-minute scheduler firing a batch that comfortably fits in four minutes is more robust than a batch tuned to just fit, because the failure mode of "slightly slower than usual" is a completed batch rather than a killed one.
What to instrument
- Attempts per completed row, as a distribution rather than a mean. A long tail tells you which validator is fighting the prompt.
- Rows left in
processingat the start of each batch. Should be zero; if it is not, your lease expiry is missing or too long. - Wall-clock per batch against your timeout, as a ratio. Anything above about 0.7 is a batch that will eventually be killed.
The summary
Money is rarely the constraint on synthetic generation — the whole 50,000-row corpus costs tens of dollars. Wall clock is the constraint, it is set almost entirely by concurrency, and the thing that will actually hurt you is a platform timeout quietly eating locked rows.