Thursday, November 10, 2016

How Did the Pollsters Get the Election Wrong?

Greetings and Salutations,

This past Tuesday, November 8, 2016, marked the end of a historic election cycle in American history.  We, the people of the United States, elected our 45th President in one of the most hotly contested battles between two very unpopular and not well-liked candidates.  Be that as it may, the people of our great nation and perhaps even the world were stunned when the "underdog" Donald J. Trump was elected over the establishment candidate, Hillary Clinton.  So how in the world did that happen when practically all the national polls showed otherwise?

While there is no easy answer, there might be some logical explanations.  First off, what exactly is a poll?  Basically, a poll is the recording of an opinion or the actual process of voting, but for the purpose of this blog, we're talking about polling in the sense of recording an opinion.  Here is where this gets really screwy!  A poll is supposed to be one question with multiple choice answers and the people being polled pick one.  However, a survey asks multiple questions over a range of issues.  So when we see what the "national polls" are saying, are they really polling or surveying.  This is an important set up to what I'm about to delve into next.

Polls, in terms of the infamous "survey of likely voters" do not ask the simple question, "Whom are you voting for?"  They ask a range of questions, which is actually a survey; I know, semantics!  But for people who conduct actual research, this is a crucial difference!  Why?  Because the question or questions being asked can skew the results!  I know, I know, hold the phone, get outta town, what you talkin' 'bout, Willis!!!?  Let's put it in a way that makes a little more sense.  If a person is asked, "Which of the of the following candidates is your choice for president," the answer will be clear and straightforward.  That is a poll.  Now imagine if you were asked several questions phrased like, "Based on the audio recording of the candidate talking about sexually assaulting women how likely are you to vote for him," or "Due to the revelation of new emails discovered by the FBI, how likely are you to vote for her," followed by a series of Not At All Likely, Less Likely, Neither Likely or Not Likely, Likely, or Very Likely, now one has to interpret the answers to draw a conclusion, in other words, a survey.  Are you with me so far?  Good!

The second part to figuring out what you have is to determine your sample.  A sample is a subset of a population that we wish to learn about.  We use samples to draw a general conclusion about the greater population because it is not always practical to conduct a census which is polling or surveying every single person in the population.  There is a lot of science behind how many people need to be included in the sample to accurately measure the results, but researchers try to make the sample representative of the general population.  With that, we are now getting to the crux of how the pollsters may have gotten this wrong.

If one samples an area that is predominantly Democrat-leaning, the results of the survey will likely favor the Democrat running for office.  Likewise, if the samples are predominantly Republican-leaning, the results will likely favor the Republican running for office.  This is what we call bias, and this occurs when particular characteristics are over-represented or underrepresented in the sample.  But here's where the pollsters failed to use cognitive reasoning.  Donald Trump was favored strongly over his Republican rivals all through the primaries by large margins.  Why then did they think the millions who voted for Trump would suddenly not vote in the general election or all of sudden vote Democrat?  This, my friends, is an example of the suspension of disbelief coined by philosopher Samuel Coleridge (1772-1834).  That is to say the people conducting polls suffered a willingness to suspend their own critical faculties and believe the unbelievable which is that suddenly all of these Republicans would not vote or choose to "jump ship" and vote Democrat.  Call it liberal media bias, call it a misjudgment, call it whatever you'd like, but it laid the foundation for one of the biggest polling blunders in decades!

How could this have been avoided?  First off, STOP FOLLOWING TRADITIONAL PARADIGMS IN A NON-CONVENTIONAL ELECTION for one!  Secondly, the L.A. Times poll which was one of the only national polls that successfully predicted a Trump win did something not used in traditional polling methods.  They asked their polled subjects one question, "How committed are they to their candidate," on a scale of 1-100 and they kept the same sample group throughout the election cycle!  BOOM!  That is called research, people!

In almost every election cycle, there is a "hidden vote" for the "unlikely" candidate.  It was there for Obama in 2008, it was there for Bush in 2000, and it was there for Clinton in 1992.  Trump had the hidden vote all along, but the suspension of disbelief caused the most intelligent of pollsters to miss what was staring them right in the face.  Plus, there are those who supported Trump that gave misleading information to the pollsters, or were just worried about how they would be viewed if they said they were supporting him.  Even director, Michael Moore, said there was a very real possibility of a Trump win months before the election.   However, the hidden vote would have revealed itself had the pollsters gathered themselves a control sample group of voters, and deferred back to them every so often during the election cycle to see if the proverbial needle changed.  A control group is the people that receive no exposure to the treatment or variables.  In this case, the pollsters would not ask the control group the survey questions, just ask them the one question the same way each time, "What is your commitment to your candidate?"  Instead, what the pollsters did in this cycle was stick to what they've always done, and that is poll a new group each time they conducted a poll after some major event.  So of course, each time the media breaks something about Trump, the needle pegged strongly towards Hillary on each new group.  This led to confirmation bias and why the polls showed Hillary touting double digit leads at one time!  The LA. Times in their masterful brilliance, however, deferred to their control group where the needle held strong with Trump showing a slight advantage because the commitment to their candidate NEVER WAVERED!!!!  Hillary's committed voters did!

Now we can sit here and say it was Trump's straight talk, his immigration stance, his economic policy, his threats to "throw Hillary in jail," or any number of shock and awe statements he's made throughout the campaign.  I'll leave that to the political pundits.  But as for the polling, the pollsters screwed up...BIG TIME!!!!  Folks, ask yourself, when was the last time a presidential candidate won the election by 14-18 points?  With all of the breaking news over the past year, when Hillary Clinton was "up" by that amount, didn't someone find that as odd?  Again, the suspension of disbelief played a key role.  Heck, it even showed Gary Johnson with 9 points at one time!  Seriously?  The numbers weren't making any sense, and even though Hillary would have seemed to be the likely elected president, the numbers were skewed too much in her direction, and that should have caught someone's eye.

So to you pollsters out there, take this as a learning experience in the future.  Remember the principals of standard deviation which is a measure that is used to quantify the amount of variation or dispersion of a set of data values.  When you start seeing major dispersion away from the norm, check your data and make sure you aren't skewing it one way or another.  If the graph to the left was the voting public, and it shifted strongly to the left or right, you may have a problem!  Voters don't normally do that!  I'll leave you pollsters to chew on that for a bit, and for those of you who voted Trump, congratulations on an astounding victory.  For those who voted Hillary, you always have 2020!  Thank you for reading, and we'll talk again.