Determining Survey Sample Size Needs

Scenario One: You want a survey sample size that minimizes sampling error

First off, we need to assess why you’re doing a survey in the first place. Some people need to understand general trends within a category or market to gauge overall preferences. This could include things like product concept screens, ad and creative testing, or buyer journey research.

In contrast, you may need to hone in on very specific metric about a population. One great example is a brand awareness study. Since these studies focus on isolating a number to use for benchmarking, you need far greater accuracy. A general read is not enough.

This gets us to one key metric to use when determining a survey sample size: margin of error.

Margin of error is how confident you can be that the metric is accurate. For instance, a margin of error of 5% means that the actual number is within plus or minus 5% of the number in your survey.

When specificity matters, like with awareness studies, we need a lower margin of error. This means a larger survey sample size. When general trends matter more, we can accept more error and therefore smaller sample sizes.

We like this margin of error calculator to back into the final error numbers.

Scenario Two: You want a survey sample size that lets you read differences over time

Another common scenario that results in asking about survey sample size is the case where you want to look at longitudinal changes. That is, you are doing research work to get a benchmark number. And, you plan to measure that benchmark over time.

Brand awareness is one typical metric in this category, but so are complaint rates, satisfaction rates, and any other measure you’re looking to improve as time goes on.

In this case, we care about something new: statistically significant differences.

Statistical significance measures how confident we can be that differences we see in our data are actually differences and not the result of sampling error. When measuring differences over time really matters, we want to increase the chances of being able to read differences. This means larger sample sizes. In contrast, when we are looking for a point-in-time measurement, we can accept lower confidence and therefore lower sample sizes.

Here’s an online calculator we like to measure statistical differences.

Scenario Three: You want to look at data from sub-populations

Now, let’s look at another common situation that pops up when looking at survey sample sizes. This is where you want to gather data from specific groups within your larger sample.

For instance, let’s say you start with recruiting dog owners. However, you then want to ask a series of questions to people who own small dogs. You now need to think about having enough small dog owners in your sample population.

If this is a scenario you’re facing, then you need a bigger sample size. Essentially, you need enough people to do a broad sweeping review of the category and then also enough within your sample to look at this smaller group.