Idea of the DayWednesday, July 8, 2009

Get Rid of Polls


Polls are as integral to the American political tradition as sex scandals or earmarks. Yet it's not clear that they serve any beneficial purpose. When a new one is published in The Times or The Post, I -- along with everyone else -- read it. But it seems to me that polls are qualitatively different from the rest of the content that fills the papers.

News organizations are supposed to provide information that holds government accountable and helps the citizenry make informed decisions on Election Day. Polls turn that mission on its head: they inform people and government of what the people already think. It's time to do away with them.

Were it not for three problems, polls would be as harmless as printing baseball scores or Michael Jackson commemorative editions.

First, constant polling uncomfortably expands the domain of democracy. There are, of course, lots of ways in which the U.S. might be able to use a little more democracy. (Think the Senate.) But the value of the referendum has its limits. (Think California.) Writers have been whining about the "tyranny of the majority" since Tocqueville for a reason: getting the input of the citizenry at regularly appointed intervals has real benefits--among them stability and reliability and the chance for a politician or policy to succeed or fail within reasonable time constraints. Poll-testing every decision, on the other hand, disturbs the balance between democratic legitimacy and democratic effectiveness.

Second, many polls are wrong. Which isn't to say that the opinions the American people express in polls are factually incorrect, though that's sometimes true. What I mean is that polls are a terrible indicator of the citizenry's actual preferences. Part of the problem is that many people have a tendency to say one thing ("stated preference") and then do another ("revealed preference"). Another part of the problem is that the public is sometimes simply confused.

My favorite recent example of such confusion pertains to cap and trade. According to one poll, three-quarters of Americans think the U.S. should regulate greenhouse-gas emissions, with a slight majority saying they would support a cap-and-trade program of the type now being considered in the Senate. But as another poll makes clear, most Americans don't even know what cap and trade is: slightly fewer than one-quarter of respondents could even identify it as having something to do with the environment. 

Third, and of perhaps greatest concern: the outcome of one poll can affect future polls and behavior. As behavioral scientists and economists are fond of pointing out--in books like Nudge and Predictably Irrational--popular behavior can snowball. Public-health campaigns emphasizing how few teenagers smoke are more effective in deterring teen smoking than those that emphasize lung cancer or bad breath. Likewise, the perception that a candidate or political position is popular today will make the candidate or position more popular in the future. As Cass Sunstein and Richard Thaler put it in Nudge, "Nothing is worse than a perception that voters are leaving a candidate in droves." Voters should be free to switch allegiances whenever they want, but they should do so for substantive reasons, not because they're following the flock.

Most everyone acknowledges the problem with polls when it comes to Election Day: exit polls are frowned upon and in some cases banned, because early ones have been shown to influence the behavior of people who haven't yet made their way to the voting booths. If we can see that it's a problem on Election Day, shouldn't we acknowledge that it's a problem the rest of the year as well?

Archives for Idea of the Day


TrackBack URL for this entry:

Comments (5)

From "Poll Position: John Zogby and the Future of Polling," published in the nation last August:

The singular achievement of polling has not been to "get at the ultimate meaning of life," as Zogby fatuously suggests, but to organize a welter of data into impressions of demographic trends that can be sold to and exploited by marketers. A majority of respondents could never be wrong, because in marketing desire matters above truth. If the "transformation of the American dream" is convincing, it's as good as money in the bank. The same calculation applies in the world of politics. In the early days of polling, corporations hired men like Gallup to help them engineer products that would appeal to the broadest swath of Americans. Though Gallup found little difference in how people think "from politics to toothpaste," his election-year surveys mainly served to test and advertise sampling methods that would garner profits in the corporate world. (Zogby outdoes him, calling voting and shopping "parallel expressions of the same mind-set," and envisioning a world in which red states and blue states have been replaced by Wal-Mart and Filene's.) But politicians understood that opinion research enabled them to craft targeted messages that could be sold just like canned soup or washing machines. As television sets became commonplace in American homes and the country eased into "life in the grid of two hundred million," as George W.S. Trow put it, the language of marketing and the language of politics merged. "No one, now, minds a con man," Trow concluded. "But no one likes a con man who doesn't know what we think we want."


This article actually missed the major flaw in survey methodology. As someone who does some market research, I am keenly aware that how you phrase a question can have a huge impact on your results, even when you are trying to obtain accurate information. Those who conduct tracking surveys often find that very small changes in verbiage can produce changes in the results that are clearly not present. Many of those conducting surveys are trying to obtain accurate and actionable data, but a large number are interested in a specific outcome and will deliberately or inadvertently structure the questions so as to produce a more desirable result. Sometimes, as with advocacy groups, it's really blatant -- I have seen "surveys" that pose such questions as "Do you believe corporations should be permitted to manufacture toxic and dangerous products that will be used by children and pregnant mothers?" The results of such "studies" will then be cited as proof that the public supports the cause in question -- and sometimes gullible reporters will cite such surveys in news accounts as if they had some value. Even when pollsters don't bias their interview questions so blatantly, they often field bad products as a result of not understanding survey methodology. This overview doesn't even address the question of sample size and composition, which confound this further. Some firms produce good results that can be replicated, but more probably don't.

Getting rid of polls is an intriguing idea. I wouldn't mind seeing a poll to see where Americans stand on the idea.

Why don't we get rid of nitwit columnists who advocate getting rid of polls?

Oh, wait -- that might violate that pesky First Amendment. Damn.


I'll agree, that is a "special" idea. It is also obviously anti-populist elitism. It boils down to a form of censorship. It stems from fears that people might be wrongly swayed, with the elites deciding how and why it is wrong. What I do find humorous, however, is the idea that you can get a better educated population by limiting information presented to them. Terrible.

Comments on this entry have been closed.