How Big Should My Survey Sample Be?

The number people expect is a percentage of their audience. The number the formula returns is a flat count, and it barely shifts between a market town and an entire country. That mismatch is worth sorting out before you commission anything.

Three inputs, and only two of them are choices

Confidence level sets the z-score: 1.645 for 90 percent, 1.96 for 95, 2.576 for 99. Margin of error is how wide a band you will accept around whatever result comes back. The third input is the proportion you expect to find, and if you have no idea, 50 percent is the safe entry, because it produces the largest sample the formula will ever ask for.

Run it at 95 percent with a 5 point margin and you get 385. Now suppose a pilot suggests the true figure is nearer 10 percent than 50. The variability term drops from 0.25 to 0.09 and the requirement falls to 139 responses. Knowing something in advance was worth 246 people.

Population size only bites when the population is small

With 50,000 people to draw from, that 385 becomes 382. With 2,000 it becomes 323. With 200 it becomes 132. Below a few thousand the finite population correction genuinely helps you, and above that it is noise. It is also why a national poll and a city poll can both sit near a thousand respondents without either of them cutting corners.

Response rate is what sets the budget

385 completed responses is not 385 invitations. At a 12 percent response rate you need to contact 3,209 people. At 4 percent, which is ordinary for cold email, you need 9,625. Then subtract everyone who starts and abandons halfway, and everyone your screening questions disqualify. Build the invitation list backwards from the completion target rather than forwards from the list you happen to have.

Comparing two groups is a different calculation

An A/B test asks for a baseline rate and the smallest effect worth acting on, not a margin of error, and small effects get expensive fast. Lifting a 3 percent conversion rate to 4.5 percent takes roughly 2,000 visitors per variant. Lifting it to 3.3 percent takes about 25 times that, since the effect being chased is five times smaller and the traffic required grows with the square of it. If the question is "did this change anything", reach for a test calculator rather than a survey one.

Using the sample size calculator

Set the confidence level, which should be 95 percent unless the decision is expensive enough to justify 99. Type the margin of error you can live with. Enter the population if you know it, and leave it blank or very large if you do not. Put the expected proportion at 50 percent unless you have prior data pointing somewhere else. The figure that comes back is completed responses, not invitations.

Your prior proportion can come from last year's survey, a pilot of 50 people, or analytics you already collect. Population size comes from a CRM export or a membership count. Once you have the target, divide by a realistic response rate to size the send list. Spread matters too, and for numeric questions such as spend or age you will want a standard deviation from a pilot rather than a proportion. If the underlying ideas are new, the probability basics are a reasonable place to start.

Common questions

Is 100 responses enough? It gives a margin of error of about plus or minus 9.8 percent at 95 percent confidence. A result of 60 percent then means somewhere between 50 and 70, which is too wide for most decisions and fine for a rough temperature check.

Why does a bigger population not need a bigger sample? Precision depends on how varied the answers are, not on how many people exist. Once the population is large relative to the sample, adding more people to it changes nothing about how much a random draw of 385 can tell you.

What if I need results broken down by subgroup? Each subgroup needs its own sample at the precision you want for it. Four regions at plus or minus 5 percent each is roughly 1,540 responses in total, not 385 sliced into quarters.

Should I use 50 percent even when I know it is wrong? Use your own estimate when it is credible, since it lowers the requirement. Use 50 percent when the estimate is really a guess, because an estimate that turns out to be too low leaves you under-sampled and the margin wider than you promised.