« Louisiana 2nd CD - 12 oppose the incumbent | Main | Pennsylvania Greens withdraw »

Why the polls aren't predictive

Over the years, the number of political polls has exploded. Thirty years ago, the only outfits who concentrated on political polling were Gallup and Harris. Now there are literally dozens of polls and polling companies. Several media alliances operate their own, several serve each political party, and many independent polling firms are in the field. The science of polling, like almost every pursuit besides love, has advanced dramatically.

Yet, reliability has never been more in question, despite the scientific progress. Why is this?

Many astute observers have been deconstructing polls in recent years. Every aspect is examined - samples, screening, adjustments, the wording of questions. Every methodology is questioned, and many times the analysis accounts for errant results. Not every poll releases their "internals," the detailed data the poll harvested to reach its numbers. Those who do make this information readily available in the spirit of "transparency" generally have more credibility - they aren't afraid to have their numbers checked.

Some polls never disclose this information. They consider it "proprietary information" - which competitors might use to take away business. There is an argument there, but it is rendered less effective by the fact that many polls DO disclose their raw findings and methodology, and they don't suffer by it.

There is one area no one talks about, though - the one statistic that NONE of the major polls disclose. They ALL conceal it as "proprietary." It is, perhaps not coincidentally, one which could explain the growing disconnect between poll numbers and actual results.

I speak of the "response rate." Never heard of it? Well, there's a darned good reason pollsters don't talk about it.

The response rate is the percentage of people actually contacted who agree to answer the poll questions. Theoretically, the higher this number is, the more accurate the results will be. Up until the early 1980s, national polls had response rates over 75%. From that time forward, the response rates began to decline steadily. It is claimed that this has now fallen to around 25%, although no one in a position to know will go on record with that.

Who has most evinced suspicion and distrust in polls over the last 25 years? It has been conservatives and Republicans. If these groups are acting on this distrust by refusing to participate in polls, there could be a disproportionate sample for any poll.

I can't prove this is happening. IF it is, it would help explain the trend of Republicans to do better in the actual vote totals than they did in the opinion polls. And it certainly seems that in recent years the polls have often erred on the side of Democrats, but far less often by overeestimating Republican support.

If my theory is correct, it also says that no amount of dissecting poll internals will ever be sufficient to adjust them to a norm. Even a sample which is statistically "balanced" by overweighting the Republican responses can't truly adjust. They were only able to poll the subset of "Republicans willing to respond to polls," and that population would be expected to vary from the larger group.

I can't prove my theory, but it does seem to be compatible with the facts. It would go a long way towards explaining the disparity between polls and real vote totals, and even eliminate conscious bias as a cause. But it's just a theory - you know, like "evolution" or "relativity" and such.

;-)

  • Currently 0/5
  • 1
  • 2
  • 3
  • 4
  • 5
Rating: 0/5 (0 votes cast)


Close

Email this entry to:


Your email address:


Message (optional):


AddThis Feed Button

Comments (6)

I'm one of those Republican... (Below threshold)
Jim Edholm:

I'm one of those Republicans who is always amazed at how far off the polls are... luckily, I don't let it disturb me because I KNOW the results will be different.

However, I have trouble subscribing to your theory about response rate being low among Republicans and conservatives. I'm dying (dieing?) for one of those pollsters to call me -- my theory is that I WANT to have my voice heard, so I would think I'm more likely, not less, to respond to a poll.

Doesn't that make sense? Why wouldn't someone who feels underrepresented be anxious to respond and have themselves heard?

I refuse to answer polls. I... (Below threshold)
Baggi:

I refuse to answer polls. I think they are damaging to our political system and turn our voting system into a race.

What does it help anyone to know the results of a poll about candidate A or candidate B?

They are especially disturbing to me in a primary race. I still remember voting for Alan Keyes in the 2000 primary and having people tell me, "Why are you voting for him? Your vote for Keyes is a vote against Bush."

Well I didn't like Bush or McCain then, I only liked Keyes. And Keyes had a lot of support, the loudest cheers, a great speaker, and yet got hardly any votes.

That's because people didn't want McCain so they once again chose, "The lesser of two evils."

I really do think polls damage our system. We should do away with them.

We don't receive polling ca... (Below threshold)
Robin:

We don't receive polling calls because our phones are cell and internet. I wonder how many others, particularly conservatives appalled by a land line phone service that includes $20/month in taxes (33% of the total bill!), are utilizing other phone systems.... and therefore are not on the lists.

"<a href="http://w... (Below threshold)
"THIS study by the Election Science Institute says exit polls in Ohio put John Kerry ahead of George Bush because those who voted for the President were more reluctant to respond to the pollsters. (Well, duh! WHO in his right mind still expects Republican voters to trust ANYbody who might work for the MSM (mainstream media)? Oh, I forgot -- the MSM!)

-- copied from THIS page

Thanks for all the comments... (Below threshold)
Jim Addison:

Thanks for all the comments. A few notes on the points made above:


I am one of those Republicans who WILL talk to pollsters. I've been polled several times by state polls, and twice by national: in 1986 and 2000. I wasn't trying to say Republicans should refuse or all those who refuse are Republicans, just that it seems to me a disproportionate share of those who refuse are probably Republicans and/or conservatives, which would affect the results of a broad spectrum of polls.


There is no doubt the move to cell/internet calling and away from landlines is causing a deterioration in the quality of the samples polls are able to draw. I'm not sure that more of those people are Republicans than Democrats or independents, though - it would seem to cut across party lines. Younger people are disproportionately excluded because of this, certainly.


I no longer refer to the "MSM." For the last couple of years, I've used the designation "fMSM," which stands for FORMERLY mainstream media.

;-)

The theory reads like "Prob... (Below threshold)
Charles_in_Texas:

The theory reads like "Probable FACT" to me since no one has EVER BOTHERED to contact me for any poll. But, I DO BELIEVE that all newspapers (with and without polls) are great for bird cage liners, wrapping fish, starting camp fires, etc.




Advertisements






rightads.gif

beltwaybloggers.gif

insiderslogo.jpg

mba_blue.gif

Categories

Monthly Archives

Wizbang Politics Blogroll

Credits

Publisher: Kevin Aylward

Editors: Jim Addison, Bill Jempty

All original content copyright © 2007 by Wizbang®, LLC. All rights reserved. Wizbang® is a registered service mark.

DCMA Compliance Notice

Powered by Movable Type 3.35

Hosting by ServInt

Ratings on this site are powered by the Ajax Ratings Pro plugin for Movable Type.

Search on this site is powered by the FastSearch plugin for Movable Type.

Blogrolls on this site are powered by the MT-Blogroll.

Temporary site design is based on Cutline and Cutline for MT. Graphics by Apothegm Designs.

Site Meter