David Butler is Emeritus Professor at Nuffield College, University of Oxford. David ran the first British Election Study survey in 1964 with Professor Donald Stokes. He is viewed by many as the inspiration for the subsequent study of British elections. David is a highly respected colleague and we continue to be grateful for his support of the BES. David’s talk published here was given at a British Government event at the London School of Economics on Thursday 20 March 2014.
” I must start with an apology. This talk is going to sound rather egocentric- but not, I hope, boastful. Over the last sixty years many people, far cleverer than me, have developed the study of elections in ingenious and important ways. But I happened to be around at the beginning. At the moment when psephology was emerging as an academic subject, I was there among those labouring in a field that was virtually untilled. So some of what I say will be embarrassingly autobiographical.
Academically, British electoral studies can be said to have begun with The British General Election of 1945 by Ronald McCallum and Alison Readman. But let us not forget the antecedents.
The systematic collection of voting statistics goes back, I think, to Samuel Lewis with his mapping of the post-1832 constituencies. Also between the Great Reform Act and the Ballot Act of 1872 there was a proliferation of local parliamentary poll books (still only partially mined by historians).
McCalmont’s listing of constituency results started in 1879 and continued until 1918. The Constitutional Year Book followed suit in 1885, carrying on until 1939. The Times House of Commons was first published in 1885 and, with the exception of 1922-24 it has continued to this day. For a while it had a lively rival in the Pall Mall Gazette series. Much more recently these statistical chronicles were trumped by that dedicated archivist, Fred Craig. He recorded every constituency result from 1832 to 1992 with the votes conveniently set out in percentage form.
As an aside, I would like to applaud a study. just published but covering the 18th century? It is the two volume Elections in Metropolitan London 1700-1850 (www.londonelectoralhistory.com). Edmund Green and his colleagues at Newcastle have spent thirty years producing the most monumental collection of election statistics yet seen in Britain. It contrasts interestingly with Joseph Grego’s lively History of ParliamentaryElectioneering from the Stuarts to Queen Victoria .published in 1892.
From the 1860s Thomas Hare, John Stuart Mill and others explored proportional representation; there was also some scholarly writing on the effects of the extension of the franchise. From 1857 onwards half a dozen articles about elections appeared in the Journal of the Royal Statistical Society. One was by H.R.Droop, the originator of the ‘Droop quota’. Another notable one came in 1896 when J.A Baines contrasted the elections of the 1890s. A later piece about the LCC elections of 1897 by my cousin, Francis Edgeworth, was, I think, the first attempt to produce a formula for the relation between votes and seats. Unfortunately it suffered from an abstruse mathematical error which the great Maurice Kendall, here at LSE, uncovered in 1950 when I drew his attention to the article. The 1906 election was the subject of a Presidential Address to the Royal Statistical Society by Sir Richard Martin. His analysis was clumsy by modern standards, but it did contain the first academic reference to ‘the swing of the pendulum’ which he describes as ‘a phrase used glibly and loosely’.He went on, ’a 10 per cent ”swing” may be regarded as the greatest likely to take place in serious politics’.
My attention was recently drawn to an 1897 letter written by Salisbury, then both Prime Minister and Foreign Secretary to Curzon, his no 2 . ‘In terms of practical politics….foreign policy is now dependent on the swing of the pendulum at home. It may be taken as an axiom that no foreign policy can succeed unless it can be completed within one beat of the pendulum.’
Forty years ago my wife chanced on a much earlier suggestion of swing. It was in an 1814 letter from Lord Brougham to the diarist Thomas Creevey, ‘A Government is not supported a hundredth part so much by the constant, uniform, quiet prosperity of the country as by those damned spurts which Pitt always used to have in the nick of time.’
The most interesting work a century ago came with the little regarded Royal Commission on Electoral Systems of 1910 when one witness quoted ‘the suggestion of his eminent friend Major MacMahon’ that, in a single member two-party situation, a ‘Cube Law’ existed:
‘If votes are divided in the ratio A:B
seats will be divided in the ratio A3:B 3’.
To that I shall return. It had a great effect on my life.
In 1915 Charles Seymour’s Electoral reform in England and Wales,1832-1885 recorded the chequered fifty years after the Great Reform Act. Seymour dealt extensively with an aspect of elections that attracted much attention in the nineteenth century– electoral corruption
But that subject had to await the 1950s for a definitive analysis. In his doctoral thesis, the remarkable Cornelius O’Leary, my first graduate student, showed what happened during the forty years after1868 when jurisdiction over election petitions was transferred from Parliament to the Courts. He demonstrated how, taking advantage of the 1872 Ballot Act and the 1883 Corrupt Practices Act, a two-judge panel heard 157 petitions from defeated candidates. 56 of them actually overturned the result. By building up case law, the judges were able to move British elections from the fictional world of Charles Dickens’ Eatanswill `and the actual horrors experienced by Anthony Trollope in Beverley in 1868 to the relative purity enjoyed in the twentieth century. In 1885 Parker’s Election Agent and Returning Officer appeared. For over a century it, and its successor, Schofield, have, with remarkably few revisions, remained the bible for party workers. Actually the last unseating for electoral malpractice occurred in 1924- in Oxford as it happens- due to the Liberal MPs’ agent putting in a false return of expenses.
Moisei Ostrogorski’s Democracy and the Organisation of Political Parties, published in 1902, represents the first major academic work on party structure and electioneering. It is a weird book full of exact observation of the activities of agents and the nature of party structures but ending with a naive appeal for the abolition of parties. Graham Wallas, lecturing here at LSE, remarked ‘One seems to be reading a series of conscientious observations of the Copernican heavens by a loyal but saddened believer in Ptolemaic astronomy.’
It was in March 1945 that Ronald McCallum had the vision to see that a British General Election was a proper subject for comprehensive record and analysis. Twenty years later he wrote to Richard Rose describing his Eureka moment. ‘I was walking up St Giles [to a committee of the infant Nuffield College]. It was a matter of amour-propre to find something before I got there. Suddenly an inspiration came to me. Keynes and all that rot. We’ll have no more of that.’ McCallum went on ‘The reference was, of course, to The Economic Consequences of the Peace, a book about which I lost my temper and kept it lost. I felt that Keynes’ short, slight and sketchy remarks about the 1918 election had been accepted because the book was, in so many other ways, celebrated. I thought that when there was an election after the war it must be photographed in flight, studied and analysed.’ McCallum went on ‘When I put my idea [to a very imposing committee] there was general enthusiasm and I was told that I had to undertake it. I shimmered and expressed doubts. I was silenced by [Lord] Lindsay in his rather hectoring way. “No, no, McCallum, you cannot have so good an idea and not go through with it.” And so it was settled in a moment of time.’
McCallum and Readman’s British General Election of 1945 has had 17 direct successors. McCallum, who described himself as ‘statistically illiterate’, asked me, a 21 year-old undergraduate, ‘to provide some numbers’ and so for sixty years I became involved in the Nuffield series.
From 1959 I recruited resourceful co-authors- Richard Rose, then Anthony King, then Michael Pinto-Duschinsky. Dennis Kavanagh, my splendid and perceptive co-author from 1974 onwards, has now taken charge of 2010 and 2015, aided by Philip Cowley. The Nuffield studies quickly stimulated by-products in works about earlier contests- Trevor Lloyd on 1880, Alan Russell on 1906, Neal Blewett on the 1910 contests, Alan Thorpe on 1931 and Tom Stannage on 1935. There have also been cognate works on Referendums and on European elections.
Elections have evolved greatly since 1945 and so have the Nuffield Studies. The authors have striven hard not to be trapped by a formula. We always agreed to start with a clean sheet but actually the format has changed little. The chapter titles and their sequence varied but the logic of the situation imposed repeated headings.
The previous parliament
Party preparations
The coming of the election
The manifestos
The campaign
The opinion polls
The press
The broadcasts
Constituency campaigning
The candidates
Retrospect
The results tabulated & analysed
One characteristic of the Nuffield Studies has been the speed of production. Ronald McCallum once instructed me ‘Your duty is to seek immortality in other people’s footnotes. Write at once so they don’t get it wrong later.’ That was before the days of citation indexes, but the Nuffield studies, full of factual detail, have fared embarrassingly
well in those misleading measures of academic significance. And the broad format of the books has been quite widely copied around the world.
The opening paragraph of my first Nuffield study in 1951 put the word psephology into print, I did not invent the word and I regret publicising it, even though it offered reviewers some eye-catching headlines. But, more interestingly, the preface also included two assertions that, happily, were falsified within the next two decades. I argued that, writing immediately after the event, it was impossible to get into the minds of the party strategists as they planned their campaigns. I also argued that it was impossible to get into the minds of the ordinary voters as they made their choice.
In 1951 it seemed to us that we could only report on what was being done publicly in the campaign. But in the 1960s Anthony King and I discovered that we could get extensive access to the leading figures in parliament and the party machines, Successive co-authors and I learnt that, during- and still more after- the battle, politicians were remarkably willing to talk about their strategies. Michael Pinto-Duschinsky was particularly skilled at writing up the interviews. As the Nuffield series became known, we found that some politicians were clearly eager to shape the historical record but most were surprisingly frank, accepting us as neither scandal-mongering or partisan. I have in Oxford thirty thick volumes, containing, notes of interviews conducted in connection with the Nuffield Studies. Many of these were with agents at the constituency and regional level, many were with apparatchiks at headquarters and many were with the key strategists and front-benchers (Tony Benn. my lifelong friend, was the most prolific source).
Secondly, as far as ordinary voters were concerned, the 1960s saw the beginnings of what has become that great enterprise, the British Election Study. There was one American work which had a notable influence on early psephology. The People’s Choice, Paul Lazarsfeld’s 1940 exploration of voting in Erie County, Ohio, stimulated the LSE’s exploration of the 1950 contest in Greenwich- as well as other constituency studies in Manchester, Bristol and Keele. In 1954 John Bonham’s Middle Class Vote broke new ground with a cumulation of Gallup data.
A later American innovation, The American Voter, led to the largest of all British
psephological enterprises. Over fifty years the British Election Study has grown and grown to the vast work currently being undertaken jointly at Manchester and Essex and Nottingham.
In August 1960 Angus Campbell and his American Voter colleagues were deploying their findings at an IPSA forum in Ann Arbor, Michigan. I was there with Norman Chester, the Warden of Nuffield College, and I suggested to Norman that we might attempt a similar study in Britain. So for ten years Donald Stokes and I shuttled to and fro across the Atlantic, producing Political Change in Britain. It was much the best and most long-lasting book with which I have been associated. Donald was the cleverest man I have ever worked with and the honours which the book has won must be attributed to him, an intellectual perfectionist; I can only claim that it would never have been finished but for my role as a moderately efficient book producer. Ivor Crewe and Bo Sarlvik took over the study at Essex in the 1970s. In the 1980s Tony Heath and his colleagues continued the work in Oxford before it reverted to Essex and elsewhere, becoming a vast and sustained attempt to explore the minds and conduct of our fellow-voters.
Let me go back to the Cube Law. In November 1949 I stumbled upon the unread Minutes of Evidence to the 1910 Royal Commission on Electoral Systems. On a clumsy hand-operated computer, I applied the Cube Law formula to the previous three general elections and found that it predicted the majority with errors of only one, three and seven seats. On Christmas night, after a family party, I returned late, and rather drunk, to my rooms to write an article for the Economist which unveiled that long-forgotten Cube Law. ‘If votes are divided in the ratio A:B. seats will be divided A3:B3’
The Economist, hearing leaks about an imminent dissolution, rushed my piece into print in January 1950. The rediscovery of the Cube Law was widely picked up and the eminent statistician, Maurice Kendall, made it part of his inaugural lecture here at LSE before turning it into a series of articles for the Observer. A month later, when I was in a BBC office, helping to prepare for the first-ever election night television coverage of the results, I got a telephone call asking if I would come immediately to Chartwell to dine with Mr. Churchill.
An invitation from the greatest living Englishman was not one to be refused. Thus it was to the Cube Law that I owe the most remarkable evening of my life. Over dinner and for three hours thereafter I sat alone with the great man. It was just a fortnight before polling-day. Imagine any party leader today being allowed by his handlers to waste a campaign evening on a totally unimportant twenty-five year old student. He asked me to explain the Cube Law but soon lost interest and spent the time….well , setting out to entertain me. He reminisced over many subjects and then stopped to listen to an election broadcast by Anthony Eden. He asked me what I thought of it. I said it might be a passable speech at the Oxford Union but was a very poor broadcast. ‘Ah! you think he should have talked down to the British people’ and he gave an imitation Ernie Bevin. ‘I comes home in the evening and takes off my boots and |I says to the wife “this ain’t like the old days…” I could go on like that for hours. Anthony couldn’t. But I have never talked down to the British people’ After telling how in 1908 he had held the attention of the electors of Dundee as he explained the complexities of Tariff Reform, he moved on to 1940 and gave me a large excerpt from his ‘blood, toil, tears and sweat’ speech.
I observed that I was only fifteen in 1940 and that it had never occurred to me that Britain could be beaten. ‘What! only 15 in 1940?’ He counted on his fingers. ‘That means you’re only 25 now. Better hurry up young man, better hurry up. Napoleon was only 26 when he crossed the bridge at Lodi.’ He sipped at his fourth brandy. ‘What a small man Napoleon was. Why he could sit on his horse at Waterloo and see all the armies of Europe spread out before him. How much bigger was that evil man in Bergtesgarten with his troops deployed from the Urals to the Channel … I hate tyranny!’ And then he rolled on into a Miltonic speech about liberty.
That was a self-indulgent anecdote. However, that episode, brought about by the Cube Law, left me with a feeling that has lasted sixty-four years: I could never be totally awed by any important person that I met.
The Cube Law, of course, is not a law. It is a statistical coincidence, and it works with variable accuracy. In a brilliant 1982 article in the British Journal of Political Science Curtice and Steed showed how by the mid-1970s, instead of a net 18 seats changing hands for every one per cent swing, as the Cube Law would suggest, only 12 seats , or even fewer, switched. That was due to the upsurge in Liberal and minor-party support and the decline in the number of marginal seats.
I mentioned the statistical appendix that I contributed to the first Nuffield Study. That task was taken over by Michael Steed in the 1960s and then by John Curtice, recently joined by Stephen Fisher. With ever more sophisticated number-crunching, they have demonstrated what can be gleaned from the regularities- and the irregularities- of the voting figures.
One of my most laborious tasks for over forty years was the totting up of election expenses. Now that job is more efficiently done under the auspices of the Electoral Commission. But I must pay tribute to the work of my 1970 co-author, Michael Pinto -Duschinsky whose British Political Finance 1830-1980 chronicles the extraordinary
reduction in the cost of constituency politics after the 1883 Act. However, since 1980 it is central party expenditure that has grown. In 1992 for the first time it exceeded the sum total of what all the constituency candidates put together spent on their campaigns. Under the Electoral Commission, set up in 2000, central outlays were at long last limited and centrally recorded.
I must briefly apologise for some omissions. This talk makes no reference to thework on European elections by Martin Westlake and others. It also omits the heroic efforts by those assiduous chroniclers of local elections, Rallings and Thrasher,. It fails to touch on the definitive studies on the redistribution of seats conducted by Ron Johnston and others, as well as the work on by-elections by Cook and Ramsden. I also leave the exploration of Leaders’ Debates to the exhaustive researches of Stephen Coleman. And I am not pursuing here the significant challenges, offered by David Denver and others, to the Nuffield argument that constituency campaigning usually makes limited difference to the overall outcome.
What I want to record today is the transformation of British electioneering since 1945. In the previous sixty years, apart from the enfranchisement of women, the most notable development was the move in 1918 to automatic registration. That changed the prime task of the agent from securing an efficient register to getting out the vote. The main local jobs remained the distribution of the candidate’s letter or ‘Address to the voters’; the setting up of public meetings; and the organisation of the ‘knocking-up’ of sluggish supporters on polling day. That continued to be the situation until the late 1950s.
At the national level there used to be extraordinarily little communication or coordination at the top. The leaders travelled around the country, visiting marginal seats and giving big speeches in time to catch the early editions. 1950 offered the
memorable picture of Clem Attlee, the Prime Minister, being driven from meeting to meeting by his wife in their old family saloon with a detective in the back seat. Churchill moved more grandly by train to address regional rallies. In that election the national campaign was fought in the newspapers, chiefly through reports of the leader’s speeches. Television, then in its infancy, gave the campaign no coverage nor, virtually, did radio.
A striking feature of contests in the first half of the 1950s was that the two sides never seemed to meet. Each proceeded on its way with utterances about its own virtues and its adversary’s faults but never seeming to come to grips with the specific points fired off daily by the enemy. There were no press conferences and no broadcast interviews or confrontations during which argument could be joined and reactions compared. It was a period when elections were furthest from that ‘myth of the trial’, latent in so many newspaper editorials, the image of the campaign as a closely argued judicial contest between prosecution (the opposition) and defence (the government) before an open-minded jury (the electorate).
1959 marked a turning point. In that year the Conservatives put on a nationwide pre-election advertising campaign. Labour, thanks to its General Secretary, Morgan Phillips, transformed the headquarters press conference from a routine administrative briefing into the centrepiece of each day’s campaigning. Labour also experimented with daily private opinion polling. But, above all, television came into its own; this was, partly through more sophisticated party broadcasts but mainly through the advent of news coverage of the leaders’ tours and of local meetings, material that had previously been excluded from the air. It was in 1959 that, for the ordinary voter, television took over from the press as the most dominant source of election news.
Over the next thirty years campaigns changed incrementally. They became more and more centralised as they coped with the coming- around 1980- of the continuous 24-hour news cycle. But the nationalisation of elections was a slow process. The formidable Iain Macleod told me in the 1960s that, while the battle was on, he almost never spoke to colleagues, let alone Central Office,. It is different now.
Nationwide press advertising during the actual campaign only arrived in the 1970s (although it had been legalised by a High Court judgment of 1951). But it was electronic developments that did most to change the scene. In the 1990s the internet took over from the telephone to provide an instant link between constituencies, candidates and London. Central offices became increasingly involved in the detail of local contests. The drafting, and often the printing, of candidates’ addresses moved to the centre. Daily or twice daily briefings were faxed or emailed to candidates and agents, giving each of them an up-dated version of the party line. Headquarters expenditure escalated. Resources and polling became focused far more intensively on marginal seats. The swiftly growing impact of Twitter and Facebook has still to be assessed. Amateur blogging may now be more influential than official literature. Parties have, with varying success, tried to improve their campaign performance by importing expertise in management, advertising, and public relations from American and Australia. New words have become established in the electoral vocabulary: photo-opportunity. sound-bite, battle-bus, spin doctor, focus group, rapid rebuttal- and many others.
Some of the changes over the last fifty years can be attributed to deliberate centralisation. But some come from been the evolution of the political and social environment. Let me list a few disconnected points.
1. The electorate is constantly being modified by deaths and coming of age. The 1970 extension of the franchise to 18 year-olds made a significant difference.
2. Social class has always been a major factor in voting. In 1962 Peter Pulzer could write: ‘Class is the basis of British party politics; all else is embellishment and detail’. No one today would enthrone class in that extreme way; in 1997 less than half of the middle class supported the Conservatives. Yet class is still important.
3. In addition to the changes in age and social structure, there have also been changes in individual activity. Voters have become more volatile. Parties have become more plural. In the parliaments of the 1950s there were on average only nine MPs not attached to the Conservative or Labour parties. `In the parliaments of this century that figure has jumped to 86. The coming of multi-party politics has altered the rules of the game in the constituencies as well as at Westminster.
4. The electorate’s involvement has changed. In 1964 the first BES survey found that 90% of the sample admitted to a party identification and 88% of these said that they felt ‘strongly’ or ‘fairly strongly’ Conservative, Labour, or Liberal. By 2005 under 20 per cent felt ‘strongly’ or ‘fairly strongly’ partisan.
5. Turnout has collapsed from the 75 per cent average of the 80s and 90s to 62 per cent since 2000. A less committed or alienated electorate may be more prone to sudden changes of mood.
6. In 1950 50 per cent of the electorate claimed to have been canvassed at home. By 1997 that figure had fallen to 15 per cent.
7. In 1951 30 per cent said they had attended a public meeting. By 2005 the figure was only 2 per cent.
The field of electoral studies, so empty when I began, has expanded exponentially, When I go to the relevant shelves in Nuffield College Library I am amazed by the number and variety of titles, sometimes, I fear, works that I have long forgotten and sometimes works that I have not yet encountered, They include impressive studies of the polls and of the media, on air and in print, as well as of grass roots activity. Their number will grow.
Psephology invites many approaches. Some people are attracted by the opportunity that ballots offer for statistical precision in an unquantifiable area like politics. A vote is an individual statement which can be counted and classified. Yet we must always be wary; a vote may represent a statement of passionate conviction, or be just a mild expression of preference for the lesser of two evils, but those counting the votes cannot do other than give equal weight to each ballot-paper. That is one reason why I am sometimes troubled by the increasing dominance of quantification. In the early 1950s I found myself described as Britain’s leading electoral statistician. In a field with only one runner, it was easy to be the leader In fact my mathematical qualifications were negligible and I was quickly overtaken by far more sophisticated friends. But also I have come to feel that numbers, although necessary, can be over-rated. The personalities of leaders and the chance of events have shaped history in ways that cannot be quantified. In seeking to understand next year’s general election I expect to be more enlightened by the findings of Kavanagh and Cowley, based on elite interviews, than by the exciting revelations that the BES will offer, based on mass interviews.
I am an elderly nerd, still fascinated by the ramifications of the voting process, here and abroad. But at the end of a life spent in observing and recording, counting and classifying, I can only claim to have achieved the Socratic wisdom of knowing that I do not know. I do not pretend to have an answer to the question ‘What decides elections?’ There are no Iron Laws. No man can jump into the same river twice; both the man and the river change. The river of British politics is changing at an ever increasing rate. The changes are not easy to understand. They must be extraordinarily disorienting for the politicians themselves. Psephologists still have a lot to do.
In the next two years my friends, working on the new British Election Survey as well as on the Nuffield history, will have to cope with patterns of behaviour never envisaged in my heyday.
Electoral Studies have a past in which it has been my luck to share. But, more important, they have a future. I shall not be around but I hope that all of you will enjoy that future. Yet one word more.
Lewis Namier described elections as locks on the canal of British history, regulating its flow. The Nuffield Studies with their focus on the campaign have led to my emphasis today on what goes on during the battle, on what decides elections, But there is a larger and more important question, which cannot be answered immediately after the battle. What do elections decide? We have known six close-run contests since 1945. If any one of those battles had been won by the opposite side, how different would Britain be today?
Let me leave you with that challenge.”