Polling skepticism

Random samples are not random and likely voters might not be so likely

The first time I saw a political poll written was in 1996 during a state senate race. I had never heard of one conducted for a legislative race, though I’m sure they had been. I had never personally received a polling call and I didn’t know about crosstabs. The only polling outfit I could name was Gallup and I thought of polls as something done for high-profile races.

Back then, pollsters got pretty close to a true random sample. They called landlines and people answered because there were no cell phones or caller ID. We had answering machines, but they were for taking calls when we weren’t home. When the phone rang, somebody answered it. If a pollster happened to be on the line, you answered their questions because nobody had ever asked them before. People didn’t get polled over and over again. The sample of people called reflected the electorate, as long as the pollster’s estimate of who would vote was correct. Weighting corrected errors around the margins

Today, very few people have landlines and even fewer answer calls from numbers they don’t recognize. Pollsters have to call a lot more people to get a large enough sample size. And since phones are essentially computers anyway, they find people online who are either paid or opt-in to respond to polling questions. The people they are reaching do not reflect the electorate or even the general public. They reflect the small sample of people who are willing to answer polls.

Today, weighting is not something to correct errors around the margins. It’s an essential part of the analysis process. Polls today are a witches brew of adding to this demographic and subtracting from that one in order to have a sample that reflects the universe pollsters are trying to test. Obviously, some pollsters are better at it than others.

And that gets to another point. I used to believe that voters were creatures of habits. Some people voted in every election from local offices to the presidency, others voted only presidential years, and still others voted regularly in even-year elections but not odd years. We could accurately gauge turnout, both in numbers and demographics, by tracking voter history.

Today, the electorate is more volatile. In 2016, Donald Trump brought voters out of the woodwork. Turnout among Republicans soared both times he was on the ballot, but plummeted when he wasn’t. Similarly, younger voters began participating in elections in numbers never seen. However, while they showed up in the 2018 midterms, their numbers fell substantially in 2022, at least in North Carolina.

Over the past 14 years, we’ve also seen changes in African American turnout. In 2008, Barack Obama drove North Carolina’s Black turnout to about 22% of the electorate. It’s been declining ever since, sinking to 18% in 2022, about two percent below their share of voter registration. I don’t know that trend will continue in a presidential year.

Finally, public opinion shifts faster than at anytime in history because of the proliferation of information, both accurate and inaccurate, in the media environment. In 1996, I read my local newspaper, The Shelby Star, and the Charlotte Observer every day. I watched local news at 6pm and national news at 6:30. I subscribed to Newsweek for more in depth reporting and took the Sunday New York Times. At various times, I took The Atlantic, Harper’s, the Utne Reader and one or two other monthly magazines. I had a constant stream of information from sources I trusted.

Today, I wake up and go to Twitter for a firehose of information from dozens of sources, some trustworthy, some not, and some I know nothing about. Stories come and go. They change dramatically from initial claims as facts emerge to alter narratives. Voters who are more casual consumers of news don’t know what is real and what is not. People and organizations on both the right and left try to increase interest by sparking outrage instead of providing information.

In this environment, public opinion moves quickly. It used to turn more like a ship, slowly and steadily over a period of time. Only shocking events like 9-ll made dramatic changes. Today, it can change in a matter of days, if not hours, often encouraged by a barrage information flowing into news feeds and social media platforms.

All of this is to say that I am tired of the breathless polling wars happening in newspapers and online. People like Nate Silver, Frank Luntz, and Nate Cohn will live and die by polls. They’ve built their careers betting on them. I’m more skeptical.

Random samples are no longer random. I have less confidence that pollsters are testing the universe of voters who will actually vote. And I have no confidence the pressing matters of today driving swing voters will be influencing them in October.

Instead, polls should be viewed as single, imperfect data points. Other points should be economic numbers and the public’s response to them. The outcome of special elections helps indicate which side has the most motivated base. The outcome of presidential primary races without opponents shows some skepticism by the party base of both parties’ nominees.

Right now, Consumer confidence is climbing. Jobs numbers are holding steady, but so are higher interest rates. Debt is increasing. Democrats are sweeping special elections for Congress and state legislatures. Abortion is still a very salient issue. Dissatisfaction with Joe Biden today does not guarantee that younger voters won’t support him reluctantly in November.

Don’t hang on the polls, but don’t ignore them, either. Look at current and historical election data. This election is going to close, no matter what polls say. The only poll that really matters, or is really accurate, is the one that includes all of the voters. We’re still a long way from then.