Choosing a survey sample is a decision about evidence, not just recruitment speed. This guide compares probability sampling, convenience samples, quota sampling, customer lists, social recruitment, and paid research panels, then provides a repeatable way to estimate cost, plan fieldwork, check representation, and document limitations.
Overview
Survey sampling methods determine who has a reasonable chance of answering your questions and how confidently you can interpret the results. The best option depends on your target population, required precision, available budget, privacy requirements, and deadline.
A probability sample uses a defined sampling frame and a random selection process. It is the strongest option when you need to generalize findings to a clearly defined population, but it usually requires more planning and access to a suitable list. A non-probability sample may be faster and more practical for exploratory research, product feedback, or audiences that are difficult to enumerate. Its findings should be described as insights from respondents rather than automatically treated as representative estimates.
The central question is not “Which method is universally best?” It is “Which method produces useful evidence for this decision, with limitations we can explain?”
| Method | Speed | Cost | Targeting control | Main bias risk | Appropriate uses |
|---|---|---|---|---|---|
| Probability sampling | Slower | Often higher planning and fieldwork cost | High when a sampling frame exists | Coverage gaps, nonresponse, outdated lists | Population estimates, formal research, trend measurement |
| Convenience sample | Fast | Low direct cost | Low to moderate | Overrepresentation of easy-to-reach people | Early feedback, usability checks, internal pilots |
| Quota sampling | Moderate to fast | Moderate | Moderate to high for selected variables | Selection within quotas may still be non-random | Balanced audience feedback and comparisons |
| Customer list | Fast if permission and contact data are ready | Low to moderate | High for known customers | Excludes noncustomers and inactive or unreachable customers | Customer satisfaction, retention, feature feedback |
| Social recruitment | Fast | Low direct cost | Low unless screening is added | Platform, network, and self-selection bias | Community research, idea generation, niche discovery |
| Paid research panel | Fast to moderate | Usually a planned per-response expense | Moderate to high, depending on available filters | Panel conditioning, professional respondents, eligibility mismatch | Targeted research when internal lists are insufficient |
Quota sampling deserves particular care. It can make the achieved sample resemble a target population on variables such as age band, region, or customer type. However, filling a quota does not make selection random. Two people in the same quota may have very different chances of being invited or choosing to respond.
How to estimate
Use a simple planning model before recruiting:
Required completes = usable analysis groups × completes per group
Then estimate invitations or recruitment volume:
Invites needed = required completes ÷ expected completion rate
For screened studies, add the expected eligibility rate:
Invites needed = required completes ÷ (eligibility rate × completion rate)
These are planning formulas, not guarantees. If you need 200 usable responses, expect only 70% of screened-eligible people to complete, and expect 40% of people who enter the screener to qualify, the working estimate is 200 ÷ (0.40 × 0.70), or about 715 invitations. Round up to account for incomplete responses, quality exclusions, and quota imbalance.
For cost, use:
Estimated recruitment cost = completed responses × cost per completed response
For a panel, the cost per completion may vary by audience difficulty, screening length, geography, device requirements, and incidence. Treat any vendor estimate as an input to verify rather than a permanent benchmark. For a customer list or social recruitment, direct recruitment cost may be low, but staff time, incentives, data handling, reminder messages, and analysis still belong in the project estimate.
When comparing methods, calculate cost per usable response rather than cost per attempted response. A cheap source that produces many ineligible, duplicate, or incomplete records may be less efficient than a more targeted source.
Inputs and assumptions
Write these inputs into a sampling plan before launching the survey:
- Target population: Define who the findings are intended to describe. Include geography, customer status, age or eligibility boundaries where relevant, and the time period of interest.
- Decision and analysis groups: List the comparisons that matter. A total sample can look adequate while leaving each region, plan type, or employee group too small to interpret.
- Required completes: Set a target for the whole sample and for important subgroups. Explain whether the number is for directional learning, comparison, or estimation.
- Recruitment sources: Record whether respondents will come from a probability frame, customer list, social channels, a paid research panel, or a combination.
- Screening rules: Ask only questions needed to establish eligibility or analysis requirements. Define disqualifications before fieldwork to avoid changing the sample after seeing results.
- Quota variables: Select variables that are relevant and reasonably measurable. Do not create more cells than you can reliably fill and analyze.
- Expected eligibility and completion rates: Use recent project experience where available. If uncertain, create conservative, expected, and optimistic scenarios.
- Quality rules: Decide how to handle duplicates, straight-lining, failed attention checks, implausibly short completion times, and contradictory answers. Apply rules consistently and document exclusions.
- Privacy and consent: Explain why data is collected, what is required, how contact information is used, and whether responses are linked to customer or panel records. Collect only what the research needs.
A practical quota sheet might include: subgroup, target proportion or count, achieved count, remaining count, recruitment source, and notes. Keep the original target beside the final achieved distribution so deviations are visible.
Worked examples
Example 1: Customer satisfaction survey
A company wants feedback from recent customers across three service regions. It has a permissioned customer list and wants 300 usable responses, with at least 100 from each region. A customer-list sample is efficient because the population is known, but the team should check whether customers with recent complaints, low engagement, or outdated contact details are less likely to respond.
If the team expects 25% of contacted customers to complete the survey, it should plan for approximately 1,200 invitations: 300 ÷ 0.25. It should monitor each region separately rather than stopping when the overall total reaches 300. If one region lags, targeted reminders or an additional recruitment source may be necessary.
Example 2: New product concept research
A product team needs 400 responses from adults who meet a specific usage criterion. It wants balanced representation across two age bands and two customer-status groups, creating four planned cells. A paid research panel may provide faster screening and quota management than social recruitment, especially if the internal customer list excludes potential customers.
The team should estimate the eligibility rate before setting a final budget. If only a small share of panel entrants qualifies, the project may require more invitations and a higher recruitment budget than a general-audience survey. The report should state that quotas controlled the selected characteristics, while selection within each cell may not have been random.
Example 3: Exploratory community feedback
A team shares a survey through a community newsletter and social channels to identify recurring problems before designing a larger study. This convenience or social sample can be appropriate for generating hypotheses. It should not be presented as a precise estimate of every potential user’s preferences. Record the channels used, dates promoted, audience exposure, and respondent characteristics so the results have context.
When to recalculate
Revisit the sampling plan whenever a core input changes. Recalculate if the target population expands, a new subgroup becomes important, the survey becomes longer, screening questions change, recruitment pricing changes, response rates fall, or the research decision shifts from exploration to measurement.
After fieldwork, compare achieved respondents with the plan. Review overall counts, quota completion, source mix, eligibility rates, completion rates, exclusion rates, and missing data. A completed quota is not proof that bias has been removed; inspect how people were recruited within each quota and whether important groups were absent.
Update recruitment options and cost assumptions when panel availability, platform rules, incentive structures, or privacy requirements change. Keep a dated version of the sampling plan and record the reason for each revision. For a repeat survey, compare the current respondent mix with prior waves before interpreting changes as shifts in opinion.
Use this final checklist before launch:
- Define the population and the decision the survey must support.
- Choose the method that balances representativeness, speed, cost, and access.
- Set required completes by analysis group, not only for the total sample.
- Estimate eligibility, completion, usable-response, and recruitment costs.
- Write screening, quota, quality, privacy, and exclusion rules in advance.
- Monitor achieved respondents during fieldwork and adjust only with a documented reason.
- Report the method, recruitment sources, quotas, exclusions, and limitations with the findings.
For the reporting stage, a structured dashboard can make quota and source checks easier; see Survey Analytics Tools Compared. If the project uses customer feedback, the customer satisfaction survey template can help connect sampling decisions with question design and follow-up.