BTW, The Polls Sucked Again As Predictive Indicators

Rate this topic

7 posts in this topic

When will the polling industry learn some real-world lessons?


Over the next few days we will read about how “well” various pollsters did in this election cycle. Giving them a letter grade I will render a C-. 


Background: The Margin of Error, MoE, in polls is a statistical function that applies to any sample set. The MoE is a measure of how well the sample set in the poll, survey, etc. will likely reflect the broader population that looks just like the sample set will provide results reflecting what the sample provided. The MoE is ONLY a function of the number of data points and the certainty threshold used. It has no bearing on the questions, the questioner, demographics, etc. It is a number’s game. Some pollsters will add a bit of error for their methodology but this is mostly about N. 


Using a 95% certainty threshold, pollsters will report the MoE based on N which is the number of participants. If N=1,000, the MoE is a little over 3%, like 3.1%. Go down to N=500, and the result is 4.4%. Up to N=2,500 and the MoE is just under 2%. 


In election polling, most pollsters are willing to accept 3% which is why we see so many polls of 1,000-1,200 people. Cost go up with N and most pollsters won’t accept a near doubling of the costs and time to raise the MoE from 3% to 2%. 


First, most polls outside the MoE likely held up in that the candidate over the MoE likely won. What will be interesting is whether the winner “covered the spread” by winning by that MoE beating margin. It is my sense that if the Dem were over the MoE, they won by less than that. And if the GOP was outside the MoE, they won by more. I beleive Blackburn in TN is the perfect example. Possibly the governor there as well. 


Second, while I am not certain in an empirical way, it is my sense that most of the polls that were in the MoE, especially those that had the Dem ahead but inside the MoE, went to the GOP candidate or were this || close, far narrower than the lead in the polls. 


Third, it will be interesting to compare the total vote count nationally with the generic ballot polls to see if the popular vote held up. As we know, there is nothing about the popular vote that matters in local and state elections but it is interesting nonetheless. 


The takeaway if my observations are correct: pollsters once again UNDERCOUNTED the GOP and GOP-leaning voter block. This happens repeatedly and can now be considered a “structural” problem with polling in the modern, partisan times. 


So for those who don’t trust pollsters, keep being skeptical because they failed again. 

Share this post

Link to post
Share on other sites
15 mins ago, makorider said:

Gillum +12 !


5 mins ago, tomkaz said:

Frackin’ CNN pollsters should be frog-marched to a swine pit and tossed in. 

Yeah I definitely wonder who they were talking to.

Share this post

Link to post
Share on other sites
1 min ago, Cayotica said:


Yeah I definitely wonder who they were talking to.

As I noted, they skewed their polls to the favor of Dems by 10-13%, higher than the +3-4% that registrations and turnout would suggest. But even with that margin reduced, they still had Gillum outside the MoE. 

Share this post

Link to post
Share on other sites
1 hour ago, Cayotica said:

Polls, polls, polls I’m so sick of polls, I’m 65 and have only been polled once in 47 years; even then the pollster hung up after the first question.

When I had a land line, and my name in the phone book,I would get polled 2 or 3 times an election season.

Since I went to cell phone only, never.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to register here in order to participate.

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.