Which Pollster to Believe?

Polls can be confusing. Let's consider two from the end of this week. According to the latest from YouGov, the Tories lead on 24%, the Brexit Party on 21%, Labour on 20% and the LibDems 19%. We're still very much in four-party territory and the crisis of the two-party system we entered into after the EU elections hasn't gone away. But if you look at Survation's latest, we find a different story. Here, Labour is on 29%, the Tories 23%, BXP 20%, and the LibDems on 19%. Who's right, and who should we take with a pinch of salt?

There are two ways of looking at this. We can compare the polling with recent findings from other companies, and we can look at the methodologies employed. Looking at the mood music coming from other pollsters, only one other has been out out recently and that was from BMG Research. It had the Tories on 28%, Labour 27%, LibDems 18%, and BXP 14%, and its fieldwork was done the week before last. Not much to go with then. One might suppose a drift away from four-party politics over time since the EU election lies a month or so behind us, and the underlying political economy that gave us the 2017 election result hasn't gone away, but equally one could counter that the tedious Tory leadership election and the total dominance of everything by Brexit ensures the parties defined by the referendum - the LibDems and BXP - remain viable options. The fact two polls support the first proposition while another provides succour for the second mean neither position can be affirmed with any confidence on the basis of numbers alone.

We're going to have to look at methodology then. The case for YouGov is of all the polling outfits, in the run up to the EU elections they were consistently the outliers. For instance, in the last round before polling day YouGov had Labour on 13%, the Tories 7%, LibDems 19% and BXP 37%. In the event they massively overestimated Nigel Farage's party (it got 30.5%) but were the closest on the other parties. Compare this to Survation (23%, 14%, 12%, and 31% respectively) and BMG (18%, 12%, 17%, and 35%). YouGov come away smelling of roses and proved outliers aren't always wrong. And yet, if we wind time back a little bit, in GB-only polling for the 2017 General Election YouGov tanked with a 42% Con, 35% Labour eve-of-poll forecast, BMG 46% vs 33%, and Survation 41% vs 40%. BMG was nowhere, YouGov closer on the Tories, but crucially Survation was almost on the nose for Labour's vote.

To make sense of this we need to remember that the EU elections are second order elections. In political science terms, these tend not to matter so much to voters vs first order elections: i.e. ones that determine the government, a general election. And so we see a number of effects: turnout is usually depressed, and punters are more likely to stray from their normal voting behaviour and vote as a protest or go for a party closer to their political heart's desire. It's something we see in every by-election, be it parliamentary or for the local council, and certainly helped explain the result in Peterborough. There is also another oft unremarked effect. We know older people generally speaking are more likely to turn out than the young, with pensioners the most likely of all. This is doubly the case when it comes to second order elections, so it tends to skew toward their proclivities. YouGov polling weights its results to recognise this differential turnout by age, hence we find their massive overestimation of BXP's vote but an accurate forecast of Labour's support. Survation tend not to weight theirs in the same way, which meant they were more sensitive to the movements of younger voters in 2017 and more or less on the money.

What does this mean for Westminster intention polling now? Well, it all depends on what the next general election is going to be like. As Boris Johnson is odd on to win the Tory leadership, we know what his strategy is: ramp up the Brexit rhetoric, dribble some drivel about patriotism, and try and remake the coalition of voters Theresa May pulled together by squeezing the Brexit Party. This is not without difficulties and, if you want to be mischievous, it more or less amounts to a 35% strategy - just enough to get over the line in the context of a split electorate. But by pursuing a polarising electoral strategy, despite the usual tricks, Johnson runs the risk of uniting voters behind Jeremy Corbyn which, among other reasons, is what happened last time. When turn out is up, the differential age effect is depressed, which favours Survation. If the public look disengaged and nonplussed, then YouGov's methodology and polling has more chance of being right. Our job then is to mobilise, mobilise, and mobilise. An election could only be a couple of months away, and Johnson is both a known quantity and completely beatable.