As regular readers of this blog will by now have come to appreciate methodology can make a difference to the results obtained by polls when they ask people how they would vote in the referendum. So how does the methodology of today’s YouGov and Progressive Scottish Opinion polls stand up to scrutiny?
One important methodological finding uncovered in last Sunday’s polls conducted by ICM and Panelbase was that it was better to weight a poll so that how respondents to a poll say the voted in the 2011 Scottish election more or less reflects the actual outcome of that contest – and not to weight a poll so that it reflects what happened in the 2010 Westminster election. Interestingly in their poll today YouGov have opted to adjust their weighting method so that it is now rather closer to that used by ICM and Panelbase.
Hitherto YouGov have weighted their data by a mixture of general measure of whether or not people support or feel close to a party (known as ‘party identification’ – and not, as I have previously written, how people actually voted in the 2010 Westminster election, though party identification tends to be closer to Westminster than Holyrood vote) while checking what that weighting produced in terms of how people said they voted in the Holyrood election of 2011. This time, however, YouGov have weighted solely by how people say they voted in 2011, though in so doing they have, unusually, included a separate weight for those who said they voted Labour in 2010 but then backed the SNP in 2011.
So we might wonder whether the narrowing of YouGov’s No lead from 30 points in their last poll four weeks ago to 20 now is accounted for by the change of weighting. Apparently only a little. According to YouGov they would have reported a 53-31 lead for No rather than 52-32 if they had kept to their previous method of weighting.
What perhaps might be a more important change is in the way YouGov introduce the referendum question. Hitherto they have done so by stating, ‘If there was a referendum tomorrow on Scotland leaving the United Kingdom and becoming an independent country…’.
That reference to ‘leaving the United Kingdom’ was thought by some to help steer people towards the No camp. This time YouGov’s introduction simply reads, ‘If there was a referendum tomorrow on Scotland’s future…’. It is certainly interesting that the seemingly more neutral wording has resulted in a narrower No lead.
But what of Progressive Scottish Opinion? Their poll has been weighted to be ‘representative of all adults’. This would seemingly mean that while it has been weighted to be representative demographically it has not, in line with the company’s past practice, been weighted on the basis of how people say they voted at a previous election, whether Holyrood or Westminster.
Innocuous though such demographic weighting might appear to be, it has had a considerable effect on the reported results of this poll. It reduces the Yes vote from 32% to 27%, and has turned a SNP lead in Holyrood vote intentions into a lead for Labour. That latter finding is at variance with every previous poll conducted since 2011. Do not be surprised if as a result the poll is greeted with a degree of scepticism.
About the author
John Curtice is Professor of Politics at Strathclyde University, Senior Research Fellow at ScotCen and at 'UK in a Changing Europe', and Chief Commentator on the What Scotland Thinks website.
