In 2012, the buzz in the polling community was about how the Republican pollsters had missed the Democratic surge. The gap between the predictions of most mainstream pollsters and those of Republican strategists was so substantial that Gov. Romney was genuinely surprised to lose the election.
Now the tables have turned. In 2014, it appears that most pollsters were guessing wrong about the composition of the participating electorate. In state after toss-up state, Democrats not only didn’t win, but didn’t come anywhere near as close as many pollsters and modelers had anticipated.
What’s wrong with the pollsters?
In fairness to the pollsters, predicting elections is one of the most challenging activities in public opinion research. Usually, when we conduct polls, we are attempting to estimate the characteristics of a population that exists. The primary challenge, from a sampling perspective, is draw a representative sample from a known (and generally identifiable) population. If one wants to know the opinions of the citizens of Tampa, for example, one uses one of number of techniques to generate a kind of master list of the entire population, then samples from that list in a probabilistic manner. If the resulting sample is more female, or older, or more white, than the census data says is true of the population, one weights the results accordingly. Technically demanding, but straightforward.
In order to predict an election outcome however, one doesn’t need a representative sample of the people of United States (or of a particular state). One doesn’t even need a representative sample of the the registered voters. What one needs, is a representative sample of the voters who actually will choose to vote in that electoral cycle. In a very real sense, that population doesn’t actually exist until the polls close on Election Day.
Drawing and weighting an accurate sample of the electorate that actually will participate involves estimates of turnout of various segments of the population. This is precisely where the challenge arises. Democrats and Republicans in the last several election cycles have disagreed about what the ultimate electorate will look like. Will it be younger or older? Will it be whiter or less white? Will there be a larger percentage of women than normal, or a typical percentage?
The actual answer to each of these questions will have much to do with the election outcome, to be sure. Likewise, the best guess of a pollster about what the participating electorate will look like, as compared to what the participating electorate ultimately does look like, will have a huge impact on the extent to which a particular poll accurately predicts the outcome.
While there are a number of challenges pollsters are facing, three fundamental factors lie behind the surprising differences in polls and their resulting predictions.
First is the increasing diversity of the electorate. Whatever the participating electorate looked like by this past Tuesday, the eligible electorate is less white this year than in 2012, part of a continuing demographic trend.
Second is the extent to which different ethnic communities divide differently between the two major parties.The strong Democratic leaning of the African-American community is long-standing. More recently, we have seen the Latino and Asian-American communities breaking more strongly toward Democratic candidates, though Tuesday’s election may show some reduction in that break.
Third is the fact that different ethnic communities tend to turn out at different rates at election time, and that this difference is greater in midterm elections than in recent presidential elections.
Taken together, what a pollster must figure out in order to make an accurate prediction of an election is the ethnic composition of the participating electorate. Because this itself involves a prediction, posters are compelled to seek the best available evidence to anticipate what the participating electorate will look like. Knowing that the two major parties are actively engaged in sophisticated voter turnout efforts, it becomes increasingly difficult to know whether the trend will be toward more or less diversity in the participating electorate in a particular election.
One thing pollsters know is that expressions of intention to vote are not necessarily reliable. There are some questioning strategies that help to improve on the accuracy of this forecast, but it is simply the right thing for respondents to say that they intend to vote, and that makes it very hard for people to tell the truth. Then there is the reality that there are people who do intend to vote but for one reason or another never get around to doing it.
Of course, the accuracy of pre-election polls is something that only candidates, political parties, activist organizations and the press really need to worry about. Polls guide strategy and investment. And polls feed the 24-hour news cycle (part of why they have proliferated so much in recent years).
What matters for us, the citizens of these United States, is the direction our newly elected leaders will take us. Interestingly, that’s actually not that hard to find out .
. . but discussing the polls tends to take precedence over discussing the policies. And that is truly unfortunate.