Trone Research and Consulting

Sample Size: The Requirements for a Reliable Study

Tom Minsel, Ph.D., Head of Research and Data Science

Recently, Trone Research + Consulting (TR+C) was tasked with conducting a quantitative study to help a client better understand where and how they should invest in their facilities. This client had a sizable customer base of more than 1.1 million customers, comprised of both individual consumers and businesses.

Data Collection

Sample data was obtained by emailing invitations to our client's customer database. As is often the case with old lists, there proved to be some difficulty in reaching customers. Some email addresses were no longer valid. Other emails were being delivered to spam or junk filters. Plus, since many of our client’s customers were businesses, sometimes the email address on file was for the person who had been in contact with our client to make arrangements but not necessarily the same person who had used our client’s services. After many interventions, we ended up with 296 completed surveys.

While consulting with our client’s executive board about the survey results, a pressing question was raised. Is a sample quantity of 296 large enough to gather reliable information on the entire client base and make important investment decisions?

Can less than 300 people represent more than a million customers?

In this case, the answer is, yes.

Statistical Analysis

Two inferential statistics useful in answering this question are the level of confidence and the margin of error. In survey research, these give us a range within which a measured observation would fall if the question used to obtain the measure was asked across 100 samples. For example, suppose we obtain 95% confidence with a margin of error of +/-5 and observe that 75% of respondents to our survey find Brand X appealing. What this tells us is that the actual appeal of Brand X lies between 70% and 80%. Furthermore, we’re 95% confident that this is the case. In other words, if we conducted our study 100 times, in 95 of those 100 times, we would observe values between 70% and 80%. The other 5% of the time, we’d observe data outside this range.

The following table details margins of error at various levels of confidence common to market research given a sample size of n=296 pulled from a universe size of 1,136,898.

Universal Size = 1,136,898  
Sample Size n=296 Margin of error +/-
95% confidence 5.70
90% confidence 4.78
85% confidence 4.18

Per the table above, a sample of n=296 will yield 90% confidence with a margin of error of +/-4.78 which, in TR+C’s judgment, is generally acceptable for market research. Looking at 95% confidence, we observe a margin of error of +/-5.70. This is also relatively decent for most market research but the acceptable margin of error varies by industry. If we were conducting clinical trials research, for example, we would desire greater precision.

To quote other research professionals:

  • The margin of error in social science research generally ranges from 3% to 7% and is closely related to sample size. (National Institutes of Health)
  • In general, the precision of an estimate is related to the square root of the sample size—in other words, to double the precision, the sample size must be quadrupled. As a general rule, sample sizes of 200 to 300 respondents provide an acceptable margin of error and fall before the point of diminishing returns. (Kevin Lyons, Lipman Hearne)

Ultimately, it is up to you to determine the margin of error with which you can live. This decision is affected by several factors that need to be considered, including the desired level of precision, feasibility and the cost of sample as well as the investment of time.

Tom Minsel, Ph.D., of Trone Research + Consulting, works with a wide array of clients, bringing diversified experience in data-driven business strategy, database analysis and marketing, predictive modeling and research design.

Scott Layne

Scott Layne

President

Scott is a senior research practitioner with more than 25 years of experience with Top 50 market research firms such as Bellomy Research and M/A/R/C Research.

Read More

He is a poised and respected industry leader with a proven track record of solving business problems for Fortune 500 companies. Scott’s extensive research experience spans consumer products and services in B2C and B2B industries, including telecom, utility, CPG, automotive, financial services, travel and leisure, retail, textile/apparel and pharmaceutical.

Our one lab advantage means one team, one call, one lab for all your testing needs, making it convenient for both you and your patients.

Kimberly Ness

SVP, Insights & Marketing

With 25+ years of experience across research, branding and marketing, Kimberly has the unique ability to transform complicated data into a simple strategic plan that achieves results.

Read More

She has led marketing efforts for global and national brands and has a specialty in animal health, working with brands like Zoetis, Boeringer Ingelheim and Mars Petcare. She is known as an industry thought leader and has been a featured speaker at Global Pet Expo, AAHA, CBI, Brakke Consulting and Fetch, a dvm360 conference.

Tom Minsel

Tom Minsel

PhD, Head of Research & Data Science

Tom is our PhD Statistician and master data miner with 25+ years in direct marketing, customer targeting, modeling and research design.

Read More

His areas of expertise include statistics, applied and theoretical measurement, psychological research, survey research, summative and formative evaluation, SAS, sampling, trend analysis and generally all things analytics. Tom oversees more than 65 comprehensive studies each year to help clients better understand their customers.