Set the precision you want and read back the sample it takes to get there. Survey margins, A/B test lifts, mean estimates, response rates, dropout and cluster design effects all sit in one panel, and the trade-off table underneath prices every extra point of precision before you commit to fieldwork.
Ask for a national poll and the quote comes back at 1,000 interviews. Ask a statistician for the bare minimum and you get 385. The second number is not folklore. It falls out of one calculation with three assumptions baked in: 95 percent confidence, a margin of error of 5 points, and a result sitting at 50 percent.
n = 1.962 × 0.5 × 0.5 ÷ 0.052 = 384.16, rounded up to 385.
The 50 percent assumption does the heaviest lifting. Variance for a percentage is p(1-p), which peaks at exactly 0.5 and falls away fast on either side. A result near 90 percent carries variance of 0.09 rather than 0.25, so the same 5 point margin needs 139 responses instead of 385. Nobody knows the answer before fielding, so 50 percent stays as the safe default. It never asks for too few.
What 385 does not survive is a second question. Split those responses by four regions and each region rests on 96 replies, worth a margin near 10 points. Break them by age band as well and the cells fall into double digits. Sizing on the headline number and then reporting the crosstabs is the single most common way a survey budget gets wasted.
Sample size sits under a square root, so halving the margin of error quadruples the sample. Not double. Four times. This is the fact worth carrying into a budget meeting, because the gap between a 5 point margin and a 1 point margin is not five times the money, it is twenty five times.
| Margin of error | Responses | Against the row above |
|---|---|---|
| ±10 points | 97 | Baseline |
| ±5 points | 385 | 4× more |
| ±2.5 points | 1,537 | 4× more |
| ±1 point | 9,604 | 6.2× more |
| ±0.5 points | 38,415 | 4× more |
Confidence level moves the number too, though less violently. Going from 95 to 99 percent lifts 385 to 664, a 72 percent surcharge for four extra points of certainty. Dropping to 90 percent saves you 30 percent and takes you to 271. Most commercial research settles at 95 because the cost curve turns steep right after it, not because the number carries any deep meaning.
The practical read. If a stakeholder asks for a 2 point margin on a 5 point budget, the answer is not a negotiation. It is a factor of 6.25 in fieldwork cost. Show the trade-off table above the panel and let the number make the argument.
The intuition says a country of 60 million needs a bigger sample than a town of 60,000. The arithmetic disagrees. Both need about 385. Once the population passes roughly 20,000, the finite population correction stops moving the answer in any way you would notice.
| Population | Responses needed | Share of the population |
|---|---|---|
| 500 | 218 | 43.6% |
| 1,000 | 278 | 27.8% |
| 10,000 | 370 | 3.7% |
| 100,000 | 383 | 0.4% |
| 60,000,000 | 385 | 0.0006% |
The correction only bites when your sample is a large slice of the whole. A 480 person company auditing its own staff genuinely needs fewer replies than an open web poll, and the calculator applies n = n₀N ÷ (n₀ + N - 1) whenever you fill the population field. Leave it blank and nothing is deducted.
One caveat worth stating plainly. The correction assumes you drew names at random from a complete list. If you emailed everyone and analysed whoever replied, you have a self-selected sample, and no correction repairs that. The people who answer staff surveys are not a random draw from the payroll.
A survey estimates a single percentage. An A/B test asks whether two percentages differ. The second job needs an order of magnitude more data, and teams who carry the 385 figure across from surveys run tests doomed before the first visitor lands.
Take a checkout converting at 5 percent. You want to catch a 10 percent relative improvement, which moves it to 5.5 percent. At 80 percent power and the usual 0.05 threshold, you need 31,234 users per arm, 62,468 in total. That is 162 times the survey figure, for a change most product teams would call modest.
Three levers move that number, and only one of them is free:
Relative and absolute lifts are not interchangeable. On a 5 percent baseline, a 10 percent relative lift is 0.5 percentage points. A 10 percentage point lift means tripling the rate. Enter the first as a relative lift, the second as points. Confusing the two changes the required sample by a factor of 400 in this example, and the toggle above the input exists to stop exactly that mix-up.
Everything above returns the count that ends up in the analysis. Four things sit between that number and your recruitment plan, and each one inflates it.
Stack all four and the gap grows fast. A cluster survey with a design effect of 1.5, a 30 percent response rate and three reported regions runs to 577 per region, 1,731 completed responses, and 5,770 households approached. Sizing that project off the 385 headline would have missed by a factor of fifteen.
Every formula here rests on the normal approximation, on independent observations, and on a single analysis run at the end of data collection. Break any of those and the number stops applying.
For hypothesis testing after the data lands, the t-test calculator and the p-value calculator pick up where this one stops, and the confidence interval calculator turns your collected sample back into a range. Everything on this page runs in the browser, so nothing you type is uploaded or stored anywhere.
Common sticking points on margins, power, response rates and the limits of the formulas.
Confidence level alone does not fix a sample size. Pair it with a margin of error. At 95 percent confidence and a ±5 point margin you need 385 responses, at ±3 points you need 1,068, and at ±1 point you need 9,604. All three assume a worst-case 50 percent result and a large population.
It buys a margin of error near ±9.8 points at 95 percent confidence. A result of 60 percent on 100 replies means the truth sits somewhere between 50 and 70 percent, so 100 works for a rough temperature check and fails for anything that has to distinguish a 55 percent result from a 45 percent one.
A poll estimates one number. A test compares two, so it carries the uncertainty of both arms and has to separate them from noise. On a 5 percent baseline chasing a 10 percent relative lift, that works out to 31,234 users per arm at 80 percent power. Larger lifts and higher baseline rates bring it down quickly.
Power is your chance of detecting a real effect. At 80 percent, one genuine winner in five goes undetected. The convention traces back to Jacob Cohen treating a false negative as roughly four times less costly than a false positive. It is a budget compromise, not a law, and 90 percent is worth the extra third of traffic on decisions you cannot revisit.
Barely. A town of 60,000 and a country of 60 million both need about 385 responses for a ±5 point margin. The finite population correction only matters when your sample covers a large share of the whole, such as 218 replies from a 500 person company.
Use relative lift when the goal reads as "improve conversion by 10 percent". Use percentage points when it reads as "move conversion from 5 percent to 6 percent". On a 5 percent baseline, a 10 percent relative lift is 0.5 points, so picking the wrong one changes the required sample enormously. The toggle beside the field switches between them and restates the target rate underneath.
Compute 1 + (cluster size - 1) × ICC. Sampling 20 households per village with an intracluster correlation of 0.02 gives 1.38, so multiply your sample by 1.38. Household health surveys often land between 1.5 and 3. Enter the ICC and cluster size in the fieldwork section and the panel applies the multiplier.
Your false positive rate climbs well past the 5 percent you set. Checking daily on a four week test pushes it above 20 percent. Fixed sample sizes assume one analysis at the end, so either wait for the full count or switch to a sequential design with adjusted boundaries.
It iterates with t distribution quantiles rather than a fixed z value. For a standard deviation of 10, a ±2 margin and 95 percent confidence, the z formula returns 97 while the t based iteration settles on 99. The gap widens as the sample shrinks and is the honest figure at small n.
No. It changes how many people you contact, not how many you need in the analysis. At a 20 percent response rate, 385 completed replies means 1,925 invitations. A low response rate does raise a separate concern, since the people who reply differ from the people who ignore you, and no sample size fixes that bias.
