By Lambert Strether of Corrente.

2020 was not a superb yr for pollsters. Guardian: “Polling trade the night time’s massive loser as 2016 debacle repeats itself“; Jacobin: “After the 2020 Election, Polling Is Lifeless“; Al Jazeera: “The landslide that wasn’t: What the elections say about America“[1]; and the Bangor Every day Information: “Susan Collins defied the polls. Right here’s what they might have gotten improper.” Actually, not less than on election night time itself, a punter who adopted the betting websites would have, accurately, gone to mattress and slept simple, in contrast to those that stayed as much as watch the networks.

This publish, nevertheless, shouldn’t be about horse-race polling, however about exit polling: Polling that doesn’t attempt to choose a winner earlier than election day — can’t we simply wait? — however that seeks to elucidate what occurred[2]. I’ll first give a brief listing of “sizzling takes,” that are even now little doubt congealing into typical knowledge. Then, so we all know the place the new takes come from, I’ll have a look at the sources of exit polling knowledge and the strategies[3] used. I’ll conclude with some ideas on how one can enhance the position pollsters play in our elections, and different methods we would come to know election outcomes. To the new takes!

Scorching Takes

Right here is an, er, random sampling of sizzling takes from the Twitter. There are numerous extra like them! Operating by the varied identities:

On Latinos:

(Word the dearth of a supply).


On Whites:

On demographic shifts typically:

Summarizing the stunning demographic ends in prose, USA Immediately, “Election 2020 exit polls: Political pundits totally didn’t predict Donald Trump’s voters“:

Protection of the Donald Trump vs. Joe Biden marketing campaign largely centered on 4 areas: girls, racial minorities, senior residents and suburban voters. Typical knowledge held that Trump can be slaughtered by all of them, thereby handing Biden a landslide.

Seems, that was nearly completely incorrect. Trump improved over his 2016 efficiency amongst Black girls (+4), Black males (+5), Latino girls (+3), and Latino males (+4). It’s nearly definitely true that minority voters turned out in far bigger numbers in 2020 than in 2016, racking up crucial votes for Biden in battleground states like Pennsylvania and Michigan, however Democratic strategists have to be questioning what it implies that Trump earned a bigger share of the non-white vote than any GOP presidential candidate since 1960.

(This occurs to be written by a Republican, however I’m positive you’ve seen the identical factors made elsewhere. All of the hyperlinks go to CNN, by the best way, and never on to CNN’s supply which, amazingly or not, goes unlisted on CNN’s web page).

So these are the new takes. Now I’ll go on to indicate why something numerical in these outcomes must be approached with warning. First, we have to perceive the sources.

Exit Polling Sources

Because it seems, in 2020 we now have not one however two[4] exit ballot sources (although one doesn’t name itself an exit ballot, for sensible functions they’re the identical. From the Washington Submit, “How is TV information going to cowl the weirdest, most fraught election in historical past? All your questions answered“:

As they’ve for almost twenty years, the massive three broadcast networks and CNN will be part of because the Nationwide Election Pool [NEP] to share knowledge collected by a agency referred to as Edison Analysis, which conducts exit polls — through each in-person and cellphone surveys of people that have already voted — to anticipate the traits inside this yr’s voters. Whereas “a handful of individuals at every community” might be permitted to evaluate the exit-poll knowledge through the day, they won’t be allowed to report it till 5 p.m. Tuesday, mentioned Edison govt Joe Lenski. Edison additionally will accumulate the precise vote tallies from throughout the nation as they’re launched by native jurisdictions.

However Fox Information and the AP left the pool after 2016 and have struck out collectively, hiring a analysis operation affiliated with the College of Chicago to assist them put together their projections. Arnon Mishkin, head of Fox’s choice desk, mentioned his group was disenchanted with 2016’s exit polling, which skewed the outcomes by capturing a disproportionate variety of youthful and college-educated voters — a lot of whom lean Democratic — and didn’t totally probe the voting sentiment of mail-in and early voters. What this implies is that for the primary time since 1988, you’ll see not one however two completely different polls of the voters as you flip the channels.

So, at a minimal, if you see a sizzling take with a chart that doesn’t cite to both supply, it is best to method it with warning (and even reject it completely, as you’d with a Covid chart that gave no sources).

So, we now have a duopoly with Edison and VoteCast. Right here Pew Analysis on Edison’s logistics:

The exit ballot is a significant operation. Edison expects to survey about 16,000 early and absentee voters by cellphone, [Joe Lenski, Edison’s co-founder and executive vice president,] mentioned, and one other 85,000 or so voters in particular person. “Between exit-poll interviewers, vote-count reporters, supervisors driving round checking on websites, and the 2 very massive cellphone rooms we’ll be working on Election Day to absorb these outcomes, we now have shut to three,000 folks working for us on Election Day,” he mentioned.

The exit ballot is extra a set of interlocking surveys than a single, uniform ballot. Apart from the cellphone and in-person parts, Edison will discipline state-specific questionnaires at 350 of its 1,000 or so polling places, along with the nationwide questionnaire all respondents obtain. The thought, Lenski mentioned, is to have the ability to ask about points that could be notably related in key states.

And now Related Press on VoteCast:

AP VoteCast combines interviews with a random pattern of registered voters drawn from state voter recordsdata with self-identified registered voters chosen utilizing nonprobability approaches. Generally elections, it additionally consists of interviews with self-identified registered voters performed utilizing NORC’s probability-based AmeriSpeak® panel, which is designed to be consultant of the U.S. inhabitants.

Interviews are performed in English and Spanish. Respondents could obtain a small financial incentive for finishing the survey. Contributors chosen as a part of the random pattern may be contacted by cellphone and mail and may take the survey by cellphone or on-line. Contributors chosen as a part of the nonprobability pattern full the survey on-line.

Within the 2020 common election, the survey is predicted to finish about 140,000 interviews with registered voters between Oct. 26 and Nov. 3, concluding as polls shut on Election Day.

So, though VoteCast is handled as if it had been an exit ballot, it isn’t, in contrast to (principally) Edison, performed on the exits of polling stations. And now to the methodological points for every.

Exit Polling Strategies

The response price drawback is widespread to all pollsters, and has the potential to vitiate the complete trade. From The Age, “Why the polls had been improper – and can by no means be proper once more“:

Within the age of the cell phone, only a few folks reply calls from unlisted numbers, and even fewer need to speak to a pollster – who, for all they know, could also be a fraudster in disguise. The Pew Analysis Centre studies that its response charges have plummeted from 36 per cent twenty years in the past to only 6 per cent now. And Pew is a not-for-profit outfit that doggedly makes an attempt to contact each sampled cellphone quantity not less than seven occasions. Business polling companies don’t have that luxurious.

.[5] That’s a really skinny basis on which to foretell a presidential election. The tiniest inconsistency between the traits of that Three per cent and people of the voters as an entire might invalidate the complete trade.

The pollsters do their heroic finest to mannequin the probably behaviour of the lots from the self-reports of some phone-answerers, however all such fashions are approximations. They inevitably introduce error. Mannequin error could also be even greater than the sampling error that goes into calculating the “error margins” which can be usually reported alongside polling knowledge. Or it will not be. Nobody is aware of however the pollsters, and so they’re not saying.

Hitherto distinctive to election 2020, an unprecedented variety of votes had been solid by mail (“67% of Republicans mentioned they deliberate to vote in particular person on Election Day, in response to a Marquette College Regulation College ballot, in contrast with simply 27% of Democrats”). Edison and VoteCast dealt with this alteration in voting patterns in another way.

First, from Edison, “NEP & Edison Analysis to As soon as Once more Conduct Exit Ballot of Document“:

The NEP’s exit ballot is the one survey that might be launched on election night time that represents the views and opinions of precise voters interviewed as they solid their ballots all throughout the nation.

Because it has since 2004, the NEP exit ballot can even embrace to make sure that all voters are represented in Election Night time protection throughout the pool’s member networks and subscribers. This yr, these polls will attain greater than 25,000 voters casting ballots earlier than Election Day.

For the primary time in 2018, NEP’s exit ballot included in-person interviews with these voting at early voting places. The approach proved extremely correct in Nevada and Tennessee, the 2 states wherein it was used that yr, and was efficiently expanded on this yr’s presidential primaries in North Carolina and Texas. For the presidential election this fall, early voters might be interviewed in particular person in eight crucial states.

“In 2018, Edison and the NEP pioneered the strategy of conducting interviews at in-person early voting websites, and as we speak, we’re utilizing that precious expertise to develop these efforts for 2020,” mentioned Lenski. “It’s merely a matter of taking our time-tested fashions and making use of them to the methods folks vote as we speak.”

So Edison is an exit poill, besides when it isn’t. Now, VoteCast, “AP VoteCast isn’t an exit ballot. It’s higher“:

In 2016, greater than 40% of the voters voted early, absentee or by mail. In contrast to the legacy exit ballot, AP VoteCast meets registered voters the place they’re, reaching them through mail, by cellphone (landline and cellphone) and on-line, utilizing a random pattern of registered voters to rigorously calibrate an enormous ballot performed utilizing opt-in on-line panels.

AP VoteCast extends past the standard battleground states. It captures the opinions of registered voters who solid a poll early, on Election Day or not all. By gathering knowledge from a pattern dimension greater than six occasions the scale of the legacy exit ballot, our survey offers higher perception into varied subsets of the inhabitants, together with Mormons, Muslims, Jews, Latinos, veterans and different teams of voters.

Now, at this level, no one can say which is extra reliable: Edison or VoteCast; it’s far too quickly for any tutorial analysis to have taken place (“Exit Ballot of Document” [POW!] vs “legacy exit ballot” [OOF!]). All we will say is that they gave completely different outcomes. From the Poynter Institute, “The AP and Fox Information say Biden has carried Arizona. Why do different networks say it’s too near name?:

Fox Information and The Related Press deemed Arizona a win for Joe Biden on election night time, making their calls three hours aside. President Donald Trump and his marketing campaign howled in protest in opposition to Fox.

Now, a day and a half later, CNN and different broadcast networks insist that whereas Arizona could also be leaning Biden, the race remains to be too shut or too early to find out the winner.

Why that disparity?

Formulation for vote counts and projections are wildly advanced mathematically and costly to create, however there’s a easy clarification.

AP and Fox pulled out of a consortium of networks after the pooled effort had produced shaky ends in 2016. The remainder of the networks stayed in, pondering the system could possibly be tweaked whereas the AP had concluded it was damaged.

The problem was whether or not the accelerating transfer to early voting and mail-in voting, advancing cycle after cycle, made conventional election day exit polls invalid. AP mentioned sure and launched into inventing a brand new methodology. The Fox Information choice desk, an AP shopper, agreed and collaborated.

Ditching exit polls, . The probably break up may be inferred by get together affiliations, the combination in a given county of those that already voted and different components.

Sally Buzbee, govt editor of AP, defined her pondering in an e-mail interview with me final week:

“We made the troublesome choice to tug out of the community exit ballot consortium. Working with NORC on the College of Chicago, we developed a brand new methodology and gear referred to as AP VoteCast, which additionally captures early voters and which has confirmed extremely correct and strong.

“We didn’t develop AP VoteCast for the pandemic: We developed it as a result of we noticed the long-term traits. Nevertheless it has proved an enormous blessing given the pandemic.

“In a pre-election webinar wherein Buzbee participated, Sam Feist, Washington bureau chief of CNN, defined why his community went one other route, sticking with the consortium and its vendor, Edison Analysis. Simplifying only a bit, Feist mentioned that he and others who stayed believed {that a} supplemental model of exit polls could possibly be constructed for the early voting and mail-in segments.

Hoo boy. “Knowledgeable estimate”? That feels like a Nate Silver-esque secret sauce (or, much less politely, a fudge issue). Nonetheless, the VoteCast shopper made the early name, which stood up. However that’s the horse-race. Will Edison or VoteCast be extra reliable with exit pollings different perform, clarification? We simply don’t know but.


As readers know, I didn’t pay a lot consideration to the polls through the common (sorry, guys, I do know you wanted the clicks). My subjective sense is that doomscrolling by ballot outcomes takes time that will be higher spent doing nearly anything, together with bowling, snooker, ingesting, smoking, and so forth. One answer to any issues polls create is “election silence“; Israel, for instance, bans polls for 15 days earlier than an election, although most different bans are expressed in hours. “Social media blackouts” (7 days) have additionally been proposed. (For myself, I’d take into account a 30 day ban for each, at the least. Then once more, why not abolish each “industries” completely?) What’s improper with having the outcomes of a horse-race come as a shock?[6]

Lastly, I’d like to precise a need that we attempt to refocus just a bit bit from image manipulation by numbers and charts to reporting by people about people. For instance:

From the Harvard Gazette, “The issues (and promise) of polling“:

Political scientist Theda Skocpol isn’t prepared to surrender completely on polling simply but, however she does assume the present course of, which regularly depends on dinnertime robocalls, “artificially constructed” focus teams, and oversimplified voter classes, wants a severe overhaul. “The entire means we take into consideration what’s going to occur politically shouldn’t be based mostly on speaking to folks or observing folks of their contexts,” mentioned Skocpol, Harvard’s Victor S. Thomas Professor of Authorities and Sociology. “It’s based mostly on these strategies of knowledge assortment, and likewise enthusiastic about the information, which aren’t working anymore.”

For Skocpol, what works is one thing she has accomplished for the previous a number of years whereas researching her most up-to-date guide, “Upending American Politics: Polarizing Events, Ideological Elites, and Citizen Activists from the Tea Get together to the Anti-Trump Resistance,” with co-editor Caroline Tervo. Collectively Skocpol and Tervo made repeated journeys to eight counties in 4 swing states — North Carolina, Pennsylvania, Ohio, and Wisconsin — .

This 2016 story by the New York Instances, “Many in Milwaukee Neighborhood Didn’t Vote — and Don’t Remorse It,” actually helped that election come into focus for me. Extra like that, please. Rather more.


[1] I’ve asserted that in 2020 pollsters grew to become political actors on the tactical degree (in search of to convey in regards to the outcome they need by making it appear inevitable). That is true within the newsroom, as Taibbi factors out:

In the previous few weeks I’ve heard from a number of well-known journalists going by struggles of their newsrooms, with stress to keep away from sure themes in marketing campaign protection usually central to their worries. There are numerous reporters on the market — most of them fairly personally hostile to Donald Trump — who’re grating beneath what they understand as relentless stress to publish materials favorable to the Democratic Get together trigger.

I grant I’ve no proof comparable pressures are going down within the workplace cultures of pollsters. Nevertheless, it will appear unusual that pollsters are proof against pressures in any other case pervasive all through the political class. This doesn’t indicate a Bond villain, however quite the same old self-censorship and slanting attributable to careerism.

[2] Sure, exit polls are utilized by the networks and different media on election day to assist assess candidate’s probability of successful earlier than precise votes accumulate. This use case could even be how exit pollsters make most of their cash; I don’t know. Nevertheless, the social goal for which pollsters exist is to, because it had been, to carry a mirror as much as the voters, so we will get some thought of who voted for which candidate, and why. And, little doubt, to handicap the subsequent race. Apparently, exit polls had been first designed to elucidate. The (extra profitable?) horse-race perform got here later. From Samuel J. Finest and Brian S. Krueger, “The Exit Ballot Phenomenon” (SAGE):

Exit polling developed within the 1960s out of a need by journalists to elucidate voting outcomes to their

audiences. Over time, it remodeled from a modest effort at CBS Information to estimate the result of the 1967 Kentucky gubernatorial election right into a multimillion-dollar operation sponsored by a consortium of tv networks and designed to undertaking race winners and clarify the preferences of quite a few voting teams. Alongside the best way, it overcame technical malfunctions, inside squabbles, and misguided calls to grow to be the centerpiece of media protection of the elections.

[3] I gained’t focus on statistical strategies. For a bracing have a look at that matter, see this remark by alert reader Terry Flynn, who writes: “TL;DR – for those who don’t appropriate heteroscedasticity in polling knowledge first then “extra knowledge” doesn’t produce higher outcomes, usually the alternative.” For heteroscedasticity, see right here. Flynn urges that “the polling organisations REFUSE to ballot utilizing appropriate strategies as recognized since 1985 due to their funding masters, laziness and concern of hanging out from the pack,” which definitely triggers my priors and implies, pleasingly, that the complete polling trade could possibly be abolished (if not revolutionized). Does anyone desires to take a second shot at explaining the complete thread — possibly utilizing one or one other of the “sizzling takes” I equipped?

[4] The U.S. Census, as a part of its month-to-month Present Inhabitants Survey (CPS), surveys voters for age, intercourse, “race and Hispanic origin,” training, poverty standing, and earnings — however not who they voted for!

[5] Pew Analysis: “The excellent news is that Pew Analysis Heart research performed in 1997, 2003, 2012 and 2016 discovered little relationship between response charges and survey accuracy, and different researchers have discovered comparable outcomes. The unhealthy information is that , and there’s no signal that this pattern goes to show round as peoples’ expertise habits proceed to evolve.”

[6] One other perform that exit polls are used for, not less than internationally, is to function a test on the election outcomes. As TDMS Analysis factors out:

The chance that our vote counts are corrupt can’t be dismissed off-hand or ignored. Pc vote counts are by no means verified by full hand counts and the vote counting software program is proprietary—hidden from view and inaccessible to the general public.

Nevertheless, all our different electoral features are so deteriorated I’m undecided that checks by exit polls are all that useful. Voter rolls are going digital, which suggests they’re by definition hackable. Voter rolls have additionally been corrupted by Republican operatives like Kobach. Democrats appear to be interposing increasingly more boundaries between the voter’s expression of intent and the precise poll, whether or not by inherently unauditable poll marking units — Stacy Abrams, although justifiably emphatic in reward of Georgia’s voter registration efforts, is, oddly, utterly silent on Georgia’s horrid digital voter rolls and poll marking units — or by vote by mail (which makes voters who don’t observe instructions properly susceptible to disenfranchisement, in addition to those that, by life circumstances, discover it exhausting to “make a plan to vote.” Voters at the moment are spoken of as having “banked” their early votes, main one to surprise what lease is being extracted by the banker; the chance value, I’d think about, of already having voted for Biden when he slipped a significant cog, or for Trump when there turned out to be video of him at Little Saint James).


Please enter your comment!
Please enter your name here