Stirring the possum

I’m a little hesitant about pursuing my disagreement with Possum over election analysis and/or prediction any further. I don’t want to have a running feud with Possum. For one thing, I have better things to do. For another, I have a lot of regard for Possum’s work generally and don’t wish us to become enemies. For a third, who cares, really? The election’s over, we’re both happy with the result, what does it matter? So I will confine myself to the following:

The purpose of analysis of electoral behaviour during an election campaign should be to answer the question: “what is going on out there?” Possum and I, like all election commentators, live in an elite bubble. We can never think like a floating voter in Mackay or Para Hills. So we use various forms of intellectual endeavour to try to understand what those people are thinking and feeling. We can then use the insights we thus gain to enlighten our readers (assuming we have any).

“Predicting the result,” in terms of guessing how many seats the two sides will win and which seats will change hands, is a parlour game. It’s lots of fun for players and spectators, but of no real significance, certainly not compared with trying to understand what the Australian people are thinking and feeling about the choices put before them by the parties. It is, of course, of some political significance whether the result is going to be (say) a Labor landslide or a cliffhanger. A Rudd government with 100 seats would have been a very different government to a Rudd government with 76 seats. But whether Labor won with 81 seats or 84 seats is of no consequence.

There are several forms of intellectual endeavour which can be used to make useful comments and predictions about the likely outcomes of elections. Analysis of opinion polls is certainly one of them. My own view, however, as I said during the campaign, is that polls are no more than broadly indicative of what is going on. People do NOT always tell the truth to pollsters, and even when they do their answers can be very ambiguous.

The classic example of this was the 1992 British election, in which every poll, including even exit polls, showed Labour winning. But Labour did not win, and it seems that people did indeed lie to pollsters. Traditional Labour voters in England defected to the Tories because (a) they didn’t like Neil Kinnock’s Welsh accent and (b) they wanted Thatcher’s tax cuts. But they were so ashamed at their own prejudice and greed that they told everyone, including pollsters, that would vote Labour. They even told exist pollsters that they had just voted Labour when they hadn’t.

Polling companies that publish their findings to one decimal place, as if it made any difference, and journalists who go into a frenzy about a 1% shift this way or that, as if it was anything but statistical “noise,” do the public a real disservice by fostering the view that polls are far more scientific and meaningful than they really are.

There are other approaches to understanding electoral behaviour and to making predictions about it. One is to try to study it on the ground. I spent the last four weeks of the campaign working in a candidate’s office, in an area far from the inner-city elite suburb where I live. I spent my days talking to voters on the phone, and hearing from the candidate what voters were saying to him on their doorsteps and in shopping centres. From that I learned that working-class voters were genuinely angry about WorkChoices, while middle-class voters were at most “concerned” about climate change, and that if there were going to be very big swings to Labor, they would come from low-income areas and not from middle-class areas.

The insights of history, sociology and political science give us the ability to interrogate opinion poll data rather than just accept them as if they had the same status as, say, meteorological data. If I see a poll which shows a national or state-wide swing large enough for Labor to win Kooyong, I can ask myself: has Labor ever won Kooyong? Answer: no. I can ask: why is this? Is it because all members for Kooyong have been brilliant and all Labor candidates have been idiots? Is it because Kooyong voters are ignorant bigots? And I can answer: No. It’s because Kooyong voters are among the wealthiest, most privileged people in Australia, and they will naturally vote for the party that promises to preserve their wealth and protect their privileges.

If I am faced with a choice between an opinion poll which says that Labor will win Kooyong, and 106 years of history and political science which says they won’t, I will go with the history and the political science, and I will conclude that the poll is wrong, or at least wrongly interpreted. For this I may be scorned as a philistine by devotees of statistical infallibility, but nine times out of ten I will be proved right.

That’s why, when I saw Possum’s table of seats on 30 September, the very first thing I said, before I got into any arguments about statistical methods or whatever (about which I freely concede I know very little), was: “I’m not sure what the purpose of this exercise is when it so obviously defies commonsense and the observations of everyone in the political process.”

The question remains: what was the purpose of producing a table showing that Labor would win Warringah, Kooyong, Higgins, Goldstein and Wannon, when everyone with any knowledge of Australian electoral behaviour and history knew perfectly well that Labor was not going to win any of these seats? If Newspoll had asked 5,000 people in Warringah who they were going to vote for, and if 60% of them had said they were going to vote Labor, that would have been one thing, and even I would probably have had to accept it. But this table was not the result of seat-by-seat polling, it was simply an extrapolation of national polling. This was going a psephological bridge too far, to produce results which were plainly absurd. And the irrefutable proof of that proposition is that Labor didn’t in fact win any of these seats.

Possum has asserted that in publishing this table he wasn’t making any predictions. And I accept this is so, up to a point. When I asked him, he specifically said that he didn’t think Labor would win Kooyong or Warringah. But when I pressed him on more seats on the list, he said: “There is an average 11.6% swing against the government in its safe seats. For that to be wrong, thousands of people would have had to be telling lies to Newspoll over a 9 month period, which I simply do not believe.” So Possum clearly believed that at the time these polls were taken, people in seats like Warringah (margin 11.3%) were intending to vote Labor. Then he said: “For every safe government seat that swings less than the average amount, others in the same category will swing more.”

Now I hate to be pedantic, but the use of the word “will” clearly makes this a predictive statement. We can quibble about whether Possum meant to say “will” rather than “would,” whether his comments were predictive or conditional or whatever. If he says that he didn’t intend that statement to be predictive, I am happy to accept that. But the real question is: if the table was not predictive, what was the point of it? What was the purpose of publishing “a list of seats which a literal interpretation of Newspoll would suggest will fall to Labor, but which I along with all other sensible people know will not.”?

Possum says that I don’t understand much of his argument. He’s quite right. I failed Year 10 maths 39 years ago, and have been saved from total arithmetical incapacity only by the invention of the calculator. My scepticism about the statistical approach to psephology is therefore due in part to my inability to take that approach even if I was so minded. I am nevertheless duly respectful of the black arts of the statistician, of which Possum is clearly a senior initiate.

But, like the blind person who develops an especially acute sense of hearing as compensation, I have developed other ways of understanding the political process, partly through academic work in history and politics and partly through close involvement with politicians (I worked for one for five years) and with election campaigns over a long period of time. As I said earlier, I think this has given me at least as good an ability to make informed comments on, and even predictions about, Australian elections as I would have been able to acquire if I had been more mathematically blessed. It has certainly given me the ability to know an absurd prediction (or non-prediction) when I see one.

I might also say in conclusion that I didn’t realise at the time that Possum’s definition of a “safe” seat was a seat with a margin of more than 6%. This is far too low. Labor won seven Coalition seats with margins above 6% on 24 November, so clearly they weren’t “safe” by any reasonable definition of the word. Seats with margins just above 6% were Herbert, Kalgoorlie and McEwen, all classic marginals. My own tables use the following definitions: 0-4% very marginal, 4-8% marginal, 8-12% fairly safe, 12-16% safe, 16%+ very safe. This difference in definitions meant that we were talking at cross-purposes for much of the time. If Possum had said “There will be an 11.6% swing against the government in seats with margins over 6%” we might have had a different debate.

This will be my last comment on this controversy. I’m sure we all have better ways of spending our time than squabbling over the entrails of this election. The sun is shining, birds are singing, Kevin Rudd is in power and all’s (more or less) right with the world. I have an air-ticket to foreign parts booked for Boxing Day. In the meantime, the detailed returns of the Russian elections are awaiting attention. Peace on earth and goodwill to all men, women and possums.