Mathfolis

Potential Problems with Sampling

Unit 3 · Collecting Data

What AP Stats asks here

Sampling problems are systematic errors that survive random fluctuation. The central insight: bias is a property of the method, not of n. Large samples shrink random variability but leave a biased estimate centered on the wrong value. AP problems test whether you can name the bias type and propose a design-level fix.

Bias taxonomy

Undercoverage
frame omits part of the population\text{frame omits part of the population}
Nonresponse
selected do not reply\text{selected do not reply}
Response
participants distort their answers\text{participants distort their answers}
Voluntary response
strong opinions self-select in\text{strong opinions self-select in}
Wording
question phrasing shifts answers\text{question phrasing shifts answers}

Bias vs sampling error

Sampling error
random; shrinks like 1/n\text{random; shrinks like } 1/\sqrt{n}
Bias
systematic; unchanged by n\text{systematic; unchanged by } n
AP Tip: Always ask 'who is missing or distorted?' Selection bias missing groups, nonresponse missing replies, voluntary response missing the uninterested, wording bias distorting the truthful.
Caution: A precisely-estimated wrong number is the most dangerous outcome. Bias narrows the confidence interval around the wrong value.
Type 1

Identify the bias type

Read the methodology, then ask whether the issue is who is in the frame, who replied, how the question was asked, or how answers were given.

Example 1
A pollster phrases a question as: "Don't you agree that the current tax rate is too high?" Which bias does this introduce? (A) Undercoverage (B) Nonresponse (C) Wording bias (D) Sampling error

Practice more of this type— AI-generated · always-new problems

Generate Problems →
Type 2

Bias vs sampling error

Random fluctuation shrinks with n. Systematic shifts do not. The two studies in the example show the difference.

Example 2
Study 1 uses an SRS of 50, gets sample mean 52 with true population mean 50. Study 2 uses a biased method on n = 50, gets sample mean 80. Both studies repeat with n = 5000 (same designs). What happens? (A) Both studies converge to 50. (B) Study 1 converges to 50; Study 2 stays near 80 because bias persists. (C) Both studies stay near the original sample means. (D) Cannot tell without more information.

Practice more of this type— AI-generated · always-new problems

Generate Problems →
Type 3

Fix nonresponse

Address the cause of nonresponse: lower the cost of replying (shorter survey, mode mixing) or raise the benefit (follow-up contacts, incentives).

Example 3
A survey of homeowners about satisfaction has a 30% response rate. Which design change most directly reduces nonresponse bias? (A) Add ten more questions to learn more. (B) Multiple follow-up contacts plus a small incentive to encourage non-respondents to reply. (C) Increase the sample size to 5000 — bias disappears with enough n. (D) Rephrase the survey to push respondents toward 'very satisfied'.

Practice more of this type— AI-generated · always-new problems

Generate Problems →