Wednesday, July 31, 2013

Lets stop taking one-off opinion polls as meaning anything at all and only look at trends.

Follow the money! Who are the main beneficiaries from public Opinion Polls? YouGov and the other Poll companies of course. That polling changed politics is unquestionably true as is, I would contend, the assertion that we are today gasping for air in an avalanche of the things. Virtually every day a new poll showing the relative Party positions is published. The laws of statistics, especially the phenomenon of "sampling error" , means that from time to time a poll will suggest an outcome that is out of line with the trends. There was one recently which showed Labour and the Conservatives equal, for example, while most polls had been showing a comfortable Labour lead. This poll was jumped upon by many commentators who wrote stories about Labour's failings or about the huge "positive" influence of Lynton Crosby on the Tories' fortunes. A day or so later the next poll showed Labour back in the lead again and the story was dropped!

The sampling error in any one poll is at least  +/- 3% which means that the latest YouGov Poll should be shown like this. The probability of Labour being on any percentage from 37% to 43% is equally likely and there is no particular extra significance in the midpoint number.

Lab 40%  (37-43)
Con 34% (31-37)
LD 10%. (7-13)
UKIP 12% (9-15)

Aside from statistical error the other reason that these polls should be treated with a pinch of salt is timing. We are not in a run up to a General Election at the moment - it will not happen until 2015. Respondents in polls are not (usually) asked how they will vote in a distant election but how they would vote NOW. It is a hypothetical question and people answer such questions differently from real ones. In particular the willingness to make a protest vote (e.g. for UKIP) is likely to be greater in a hypothesis than in reality. But while UKIP's position is overstated because it is not for real at the moment it may be understated because some polls do not "prompt" for the Party. A prompt is analogous to a ballot paper. If you show a respondent a list of parties and ask him which he would vote for that is a prompt. If you ask him unprompted which he will support that will give a somewhat different result for the smaller Parties such as UKIP or the Greens. Comparing fully prompted polls with unprompted polls ( it happens) is not comparing like with like.

Hypothetical opinion polls are much more likely to be affected by recent events than a real election. So, for example, outcomes fluctuate at the time of Party conferences but generally return to trend within a few weeks. Leaders' responses to current news issues also cause poll variations. And Party stunts - for example the recent Conservative initiative to have a mobile advertising hoarding threatening illegal immigrants - may give, as was intended, a short term boost to Tory fortunes. 

Polls between elections are not unimportant but individual poll results should not be analysed as minutely as they are and politicians should refrain from comment on anything but trends. What this means is that if over time (say a couple of months) polls consistently point to a particular outcome or change then that is worth talking about. Trend analysis ignores outlier or freak polls and looks for consistency. When this is coupled with qualitative research (focus groups etc.) which gets under the trend numbers and explains WHY the trend is happening then you may have something truly useful. 


No comments:

Post a Comment