Polling, Debates and the Future of American Presidential Politics: The Mold is Broken

Polling, Debates and the Future of American Presidential Politics: The Mold is Broken

The common (and largely valid) criticisms of what are called presidential “debates” are so familiar and so well-articulated by other thoughtful observers that I see no point in rehashing them here. I’ll simply note what anyone who understands the meaning of the word “debate” already knows: presidential candidate debates are much more akin to a reality TV show than debates. Indeed, I think in some ways they are just like reality TV (which is very unreal indeed). They are like “The Bachelor,” right down to the scurrilous question about open marriage asked of Newt Gingrich at the opening of the South Carolina debate in early 2012.

The problem with the format is compounded as the number of candidates increases. If we imagine that a two-hour broadcast actually contained 120 minutes of interaction with the candidates (which, of course, it doesn’t), a debate between two candidates grants each a full hour for questions posed, answers given, answers rebutted, and opening and closing statements. With just five candidates, that number drops to 24 minutes per candidate.

With 10? Twelve minutes.

Twelve minutes to talk about everything from family background to fundamental beliefs to immigration to Iran to Syria to China to trade to climate change to unemployment to government regulation to abortion to . . .

But set all of that aside. At least we had the chance to hear something about some issues from the top-tier candidates.

But wait . . . were they the top-tier candidates?

There’s no good reason to believe that they were.

Fox News picked a number (10 . . . why not eight, or five?) and then picked a standard: the composite average of a set of recent national polls.

Does anybody actually read the methodological disclosure on these polls?

Take the disclosure on Fox News’ own poll:

Results based on the full sample have a margin of sampling error of plus or minus three percentage points.

If you read the fine print on large national sample surveys like this one, you’re used to these numbers.

What it means, in practice, is that an individual candidate’s percentage of support could be anywhere from 3% higher to 3% lower than the percentage reported in the poll . . . and that we are 95% sure of that. That means, as well, that there is a 5% chance that the candidate’s support is even higher or lower still.

But I’ll stick with 95% confident. Those are good odds.

The candidate who barely got on stage last night was Governor John Kasich of Ohio, who had the 10th highest level of support, a whopping 3% of respondents.

That means that Governor Kasich’s support is not statistically different from 0%.

Put differently, it means that there was no valid statistical reason to assert that Governor Kasich has more support than Rick Santorum (2%), Carly Fiorina (2%), Rick Perry (1%), Bobby Jindal (1%), George Pataki (less than 1%), Lindsey Graham (less than 1%) and Jim Gilmore (less than 1%).

It’s even worse than that.

That +/- 3% figure is based on total sample size (just over 1,300 registered voters). It’s that large sample (that is large by standards of national polls) that provides such a small margin of error.

But the poll question about support for the individual Republican candidates was answered by less than 500 respondents (which makes sense; this is the number of respondents who indicated they would likely participate in a Republican primary). That means that the actual margin of error is closer to +/-5%.

Which means, dear reader, that with the exception of Donald Trump and Jeb Bush, there was little or no statistical justification for including or excluding any of the other candidates. The support for George Pataki, Lindsey Graham and Jim Gilmore, each of whom garnered less than 1% support, was statistically indistinguishable from the support the candidate in third place received (Governor Scott Walker, at 9%).

I’m only a little unfair here. Fox News used the average of five recent national polls, and that does buy some additional confidence in the results (though how much is more complicated than one might at first assume).

So what we watched Thursday night wasn’t a “debate,” and it wasn’t between the top 10 candidates, even in terms of public support.

We watched a largely accidental assemblage of candidates more or less arbitrarily chosen to accommodate a format that doesn’t work anyway.

This mold is broken. It has been broken. We need a better method for learning about the men and women who seek to be the most powerful individual on the planet.

That much really isn’t debatable.

Photo Credit: Scott Olson, 2015 Getty Images

Leave a Reply