Polling won’t tell you who will win in November, but it’s not meant to

President Biden’s reelection campaign is not going particularly smoothly, not that political campaigns ever really do. But his is particularly rough, with critics and allies suggesting that he might or will step aside to improve his party’s chances for success. Nervous Democrats look at polls and see former president Donald Trump running even or leading, and they imagine some unidentified savior who might quickly and permanently turn that around.

Speaking in New Hampshire on Monday, Biden offered one argument against that (wildly unlikely) idea: Polls aren’t reliable. Polling had changed, he claimed, and is “not nearly as capable as it was before because you’ve got to make 6 zillion calls to get one person on their cellphone.”

This is a variant on the old “they only call landlines” argument for ignoring the polls and is only slightly closer to the truth.

So let’s use the occasion of Biden’s comments to do three things. First, let’s establish that polling is an effective way to measure public opinion. Second, let’s clarify that does not mean that a poll conducted today will accurately capture who will win the presidential election. And third, let’s further clarify that even the last polls conducted before this year’s election will almost certainly show no more than who is more likely to win.

Those three things might seem contradictory, but they are not. If you use a paper map to plan your route to your destination, you’ll have a good sense of how long it will take. You should not, however, assume that it will provide you with a Google Maps-like estimate of your arrival time to the minute. It’s not real-time, for one thing, and it’s simply not designed to be as precise.

This brings us to the first point.

It has become popular in recent years to wave away all polls as inherently dubious or incorrect. Polls, of course, tell you that they are not accurate to the percentage point; every real opinion poll includes a margin of sampling error that acknowledges the uncertainty about the top-line number.

This isn’t a cop-out; it’s statistics. Polling is a scientific endeavor that also requires assumptions and mandates certain constraints. You can’t survey everyone, meaning that you need to figure out how to get the population you do survey to look as much like the overall target population as you can. In other words, if you are trying to figure out who will win the presidential contest in Michigan, your poll needs to reflect the likely voter pool in that state by age, gender, race and (as was made very clear in 2016) education.

This article is not going to rehash how all of this works. I wrote an FAQ back in 2019 that covers most of it, including what’s involved in Biden’s comment about phones. Our polling team wrote an explainer in 2020. But even those are a bit outdated.

Pollsters, like everyone else, are aware that technology changes and implement ways to use that technology. See Pew Research Center’s walk-through of how its polls are conducted, for example. Or see Pew’s explanation of how polling has changed, including the graph indicating that a third of national pollsters changed methodology after 2020. That’s probably good, since 2020 polling was unusually not-good. That link, by the way, is a reminder of an important check on the process: Pollsters self-evaluate, and use those evaluations to improve and shift. Sometimes, they learn that they’re hitting the desired mark. (As polls did in 2022.)

Again, the important consideration is what you expect polling to do. If you want it to predict the winner of a 50-50 political race, you will probably be disappointed. If you want it to give you a sense of how much support a policy has — evenly split, say, or overwhelmingly supportive or opposed — you’re getting much closer to the mark.

That’s what polls do best: measure opinion within certain ranges. Unfortunately, elections are generally the opinion we seek most often and, again unfortunately, we’re doing so in a moment where elections are often much closer to the “50-50” space than the “overwhelming” one.

Bringing us to the second point.

Biden is very much correct that current polling showing Trump with a lead shouldn’t be considered immutable. (The Washington Post’s current average of polls has Trump up by one percentage point.)

538’s G. Elliott Morris posted a chart on social media that shows how polling conducted as of March 12 in previous presidential election years corresponded to the actual result. From left to right, we see the projected margin in the polling average on that date, bigger deficit for the Democrat to bigger advantage. From top to bottom, the actual eventual margin, from bigger loss to bigger win.

There are five years in which the Democrat leading at this point went on to win more votes. There are three in which the Democrat won more votes despite trailing at this point. There are three in which the Democrat was trailing and lost. And there is one — Jimmy Carter in 1980 — where the Democratic candidate had a big lead in March and lost.

Consider just the years in which Democrats were trailing (as some polls suggest Biden is). Those split 50-50 between eventual vote-margin wins and losses!

There’s another aspect to this, of course: Two of those Democratic “wins” were only in the vote margin. Al Gore trailed at this point in 2000, won more votes nationally and lost the election. Hillary Clinton led at this point in 2016 and won more votes nationally by a lot. You know what happened next.

And so we have reached the third point.

It admittedly seems like a cop-out to throw up our hands and declare that general election polling isn’t going to give you the answer you seek. But if the answer you seek is “who will win and by how much,” it won’t. If, instead, the answer you seek is “How likely is it that a candidate will win,” then polls can get you closer to your destination.

In 2016, the final 538 projection showed that Clinton had a 71 percent chance of winning. This was not the polling average, a mathematical formula the site (like others) uses to figure out what various polls say about a race. It was, instead, an analysis that looked at state-level polls and determined how often the errors in those polls would lead to one result or another.

That 71 percent was smaller than other projections, but it was still viewed as prohibitive in Clinton’s favor. That’s in part because “71 percent” is hard to evaluate. In 2019, I made this little game that allows you to see how often something with 71 percent odds of happening doesn’t.

How much money can you make?

If you hit a red block, congratulations! You won, and so did Clinton. In 2016, though, the 29 percent odds hit.

Let’s jump forward to 2020, when the final polls suggested that Biden would win, and he did. The final 538 average indicated that Biden would get 53.4 percent of the vote. But there was uncertainty; the data suggested decent odds that Biden could end up with as little as 50.9 percent of the vote.

He got 51.3 percent.

The estimates of where the electoral college would end up were similar. The final estimate had Biden at 348 electoral votes, though the range of plausible possibilities went as low as 267 — below the upper limit for Trump. Sure enough, Biden got 306 votes.

More importantly, Biden’s victory depended on his winning the three closest states he flipped by less than 43,000 votes. Take those votes away — 0.05 percent of his total — and Biden and Trump each land at 270 electoral votes. And that was in a contest in which Biden had a relatively robust lead coming into Election Day.

One response here will be that the polls were wrong, of course. But 538’s estimate gave Trump a 1 in 10 chance of winning. See how often you get a gray square on our revised game, below.

Polls said one candidate was likely to win, and he did. Even once polls close in November, we’re not likely to see data that gives us much more certainty than that.

Scott Clement and Emily Guskin contributed to this report.