Short answer: around 11,000 conversations, or 44,000 labelled turns, for a 15-class per-turn intent task. That is not a round number chosen for comfort. It is the smallest corpus on which the result you are trying to measure is still distinguishable from noise.
Most people size a dataset by asking how much they can afford. That is the wrong end of the problem, because for synthetic data the money runs out long after the statistics do.
Size the test set first, then work backwards
Your dataset size is decided by one question: how small is your smallest class in the test split?
At 44,000 labelled turns with a 15-label codebook and a realistic skew, a 6,000-turn test split leaves roughly 300 instances in the rarest class. Three hundred gives you a Wilson 95% confidence interval of about ±4.9 percentage points on per-class recall.
That number matters because of what you are comparing. If you expect your hard classes to land around 0.65–0.80 and your easy ones around 0.85–0.92, a ±4.9pp interval separates those two bands. A ±9pp interval does not — the bands overlap and your headline finding evaporates.
| Conversations | Labelled turns | Smallest class (test) | Verdict |
|---|---|---|---|
| 200 | 800 | ~40 | Pilot only |
| 1,500 | 6,000 | ~300 | Probe the pipeline |
| 11,000 | 44,000 | ~2,200 train / ~300 test | Recommended |
| 50,000 | 200,000 | ~10,000 | Don't |
Going from 6,000 to 12,000 test turns buys you about 1.4pp of precision and costs 6,000 turns of training data. That is a bad trade.
Why not just generate 200,000 turns?
Because four separate ceilings arrive before the money does.
Human review does not scale. Whatever audit you commit to is a fixed number of turns a person can actually read — call it 1,500. Auditing 1,500 of 44,000 turns is 3.4% and defensible. Auditing 1,500 of 200,000 is 0.75%, and the first question anyone asks is why you bothered.
Compute caps you harder than money. A real evaluation is not one fine-tune. It is a baseline, a context arm, a k-ablation, a noise ablation, a dialect slice, a learning curve, and seed replicates — call it 12 to 20 runs. At 44,000 turns that is a weekend on a free T4. At 200,000 it is a month of session timeouts, and the first thing cut is the ablation, which was the entire research question.
Label quality has a ceiling. Inter-annotator agreement on a 15-class Arabic intent codebook tops out around 0.88–0.92. Past that ceiling, extra rows do not buy extra signal; they buy extra rows that sit near the same confusion boundaries.
Near-duplicate pressure gets worse, not better. Deduplication at a 0.85 MinHash threshold over a finite seed grid saturates somewhere past 25,000 conversations. Beyond that point, what survives the filter is more homogeneous than what came before it. You are paying to make the corpus narrower.
Turn the size into a result
If you are worried that 44,000 sounds small, publish a learning curve: train on 25%, 50% and 100% subsets and plot the metric. It costs you nothing you were not already going to run, and it converts "only 44k?" from an objection into a defended design decision — you can point at the flat section of the curve and say the extra data was not buying anything.
What this does not tell you
This arithmetic assumes per-turn classification with a closed label space. A generative fine-tune, a retrieval task, or a domain with far more classes will size differently. The method transfers even when the number does not: decide what effect size you need to detect, compute the test split that detects it, then size the corpus around that.