Do numbers lie?

Am I the only one who is seriously concerned about the durability of this support for Yes in the polls? I see the numbers. The numbers tell a very positive story. But they don’t tell the whole story. I look at the words as well. And the words offer no assurance about the robustness and resilience of this supposed 12 point lead for Yes. Incidentally, that’s 12 percentage points and NOT 12% as proclaimed by the headline. See how numbers don’t always tell the truth, the whole truth and nothing but the truth?

The claim that ‘the numbers don’t lie’ is one that we’re hearing a lot at the moment from Dave Thompson and his Acolytes for Independence. They repeat the same arrangement of figures over and over like religious zealots handing down the unchallengeable word of their god. But there is no thoughtful analysis involved. It is a case of starting with the conclusion and working backwards until a pleasing arrangement of numbers is found which makes that conclusion appear sound and then calling it ‘proof’. At which point, no further thinking is permitted. Any who question the conclusion are dismissed as fools who don’t understand how it all works. Or they are fobbed off with a smug and smirking reminder that ‘the numbers don’t lie’.

Canadian-American cognitive psychologist and linguist Steven Pinker sums up the paradox nicely.

Narratives without statistics are blind, statistics without narratives are empty.

Numbers themselves don’t lie. Feed in the same numbers and operators and arithmetic will always give the same answer. Two plus two will always equal four – unless you add some words that alter the context. Then you get endless debates about whether two plus two really does always equal four. It does if you only look at the numbers and forget the narrative. It is the context provided by the narrative which creates the impression of some doubt about the arithmetic.

I have purposefully chosen an illustration which seems to suggest that Dave Thompson and the cunning plan cult are correct in claiming that the numbers don’t lie. But what the illustration actually demonstrates is that context is crucial. Narrative context – language – can be added to a simple calculation to cast doubt on the arithmetic. By the same token, a complex calculation may be presented absent the narrative context necessary for proper understanding. Numbers can lie if you want them to. Or if you let them.

It is essential to examine not only the numbers but also the narrative – the context – in order to arrive at a good understanding of what it all means. Thompson and his cronies use the tricks of political rhetoric to make the numbers deceive. When you listen to Dave Thompson reeling off his practised spiel, bear in mind that you are listening to a politician, That may be all the context you require.

I am not, of course, suggesting that the polling figures are contrived to deceive us. I am, however, warning of the danger that we may deceive ourselves. It’s not that the numbers lie. It’s just that to take those numbers at face value without the context provided by narrative may be to let ourselves be misled.

The human mind isn’t a calculator. It’s more like a word-processor. It can calculate. But mostly it writes stories. It accounts for observation and experience not with arrangements of mathematical elements leading to an inevitable and unchallengeable conclusion but with arrangements of ideas that can be interpreted in various ways. Numbers – mathematical ‘proofs’ – may be used to narrow the range of possible interpretation. Or to lead interpretation towards a particular conclusion. But the numbers on their own can tell us only what that particular arrangement of numbers must always tell us regardless of whether what it is telling us is relevant to the story, or helpful in our understanding of the story.

Context is king!

A twelve point lead for Yes in the polls is what the entire Yes movement wants to see. And that may be the root of the problem I’m trying to indicate. We may want it too much. There is a risk that we welcome such positive indications so wholeheartedly that we ‘forget’ to take into consideration the narrative. We fail to put the numbers into context. We let the numbers lie to us because the lie is appealing. And/or because the truth is disturbing.

We have to look at the words as well as the numbers. I am inclined to attend to the words more than to the numbers. Because I know that the numbers on their own don’t lie. They just may not be telling the truth we want to think they’re telling. There is not the slightest doubt that 56 is 12 more than 44. Those numbers, thus arranged, don’t lie. They are incapable of lying. Maths doesn’t lie. People do. Maths cannot be deceived. People can. If it is understanding we seek, we must analyse the words. Analysing the words is more important than adding up the numbers. The numbers will always add up to the same total. The words can lend a range of meanings to that total.

Am I labouring the point? Damned right I am! Because it is necessary. However innately clever or well-educated the person they are still human and subject to the frailties that beset all humans. Among which are certain biases when regarding and interpreting numerical data – such as denominator bias and recency bias. We must be constantly, incessantly reminded of these biases if we hope to compensate for them in our analysis.

When I read this article about the most recent favourable polling for Yes in The National I naturally noted the numbers. I got that with Yes on 56% and No on 44% there is a 12 point difference in favour of Yes. I don’t have to think too much more about that. Assuming it is a “serious” poll as we are assured it is then I am content to accept that the numbers – 56 and 44 – are ‘correct’ within the context of scientific polling. Accepting these as the inputs, I don’t need a calculator to figure out that the difference between these two numbers is 12. It is therefore ‘true’ that Yes has a 12 point lead. But what does it mean? What more needs to be said in order to more fully understand the reality and the implications? To get that we must look at the words. The qualifiers. The narrative.

I note that the headline includes the words “Yes soars”. This description seems to be de rigeur regardless of the trend indicated by the numbers. If we afford a previous but recent poll the same credibility as we have allowed this one then Yes has ‘soared’ from 58% to 56%. It’s an ‘idiosyncratic’ use of the word ‘soar’. But this is a newspaper. We have to expect a bit of sensationalising. What we don’t have to accept is the reality as portrayed by the sensationalising language. If the numbers are to be believed then Yes hasn’t ‘soared’ by any standard dictionary definition of that term. To claim that it has is to disseminate a falsehood.

But is it a lie? On further consideration of the language I note that no start point is identified for the claimed ‘soaring’ of Yes support. The ‘soaring’ claim may be quite truthful, if still misleading, depending on what the take-off point is and what extent and rate of increase is held to constitute ‘soaring’. See how important context is?

I also note that The National’s account emphasises the difference from No rather than the headline figure for Yes as has previously been the all but invariable case. Given what has been set out above it’s easy to understand why.

These may seem trivial points. But it is important to be aware of the conventions of newspaper story-telling. After all, our purpose is to analyse the story of this poll for ourselves and not to let some newspaper do the interpretation for us. Having made this point we can move on to something more substantive.

As I said at the outset, I am concerned about the robustness and resilience of the Yes lead suggested by the poll conducted by JL Partners for Politico. To get a sense of that it is necessary to look at the narrative provided by James Johnson – whose credibility is given a bit of sheen by mention of the fact that he used to be Theresa May’s pollster. What does he have to say about the result of his poll? From The National,

According to James Johnson that boost for yes is partly because swing voters “loathe” Boris Johnson, and partly because they believe the Scottish government has handled the pandemic, the economy, the NHS and schools better than the rest of the UK.

My immediate reaction is to ask “Is that it?”. If this is the explanation of the Yes lead then what does it say about how robust and resilient that lead is? I am not reassured!

This is a matter I came at from a different perspective in a recent article. Here, I am more interested in what reasons there may be for having confidence in this lead rather than whether the explanations for the lead constitute good reasons for voting Yes. What strikes me is that none of the explanations James Johnson derives from his poll relate in any way to independence campaigning. Which is not surprising as there has been no real independence campaigning for many years. The stuff about the SNP’s record in government is excellent news for a political party concerned with winning elections. But it only tenuously and tangentially relates to the constitutional campaign rather than the campaign for the 2021 Holyrood election.

Yes! The fight to restore Scotland’s independence absolutely depends on the SNP not just winning that election but winning it at least as decisively as the poll suggests. The problem is that while that SNP election win is necessary, it is not sufficient. It is necessary for the fight to restore Scotland’s independence. But it is not sufficient in itself to take that fight forward. The poll tells us that the SNP is popular and it tells us why. But neither of these things are enough to be confident about the support for Yes. If that support is dependent on the personal popularity of Nicola Sturgeon, for example, then it disappears if and when the popularity does. And that can happen all too easily.

That 12 point lead for Yes is only as resilient as the mass popularity of a politician and the electoral approval for a political party. Both of which are vulnerable to the kind of attack that is the speciality of the British state. The lead is not robust in its own right. It is not, as far as can be discerned from the polling, founded in any firm conviction about the need to restore Scotland’s independence. If independence only seems like a good option when compared to Boris Johnson then what happens when Boris Johnson goes the way of most British Prime Ministers?

That Yes lead could only be considered robust and resilient if the indications were that it was based on people being persuaded that independence is a worthy end in itself as well as being the means to the end of getting rid of Boris Johnson and ensuring the good governance of Scotland.

---