Hatrack River Forum: Do you trust polls?

This is topic Do you trust polls? in forum Books, Films, Food and Culture at Hatrack River Forum.

To visit this topic, use this URL:
http://www.hatrack.com/ubb/main/ultimatebb.php?ubb=get_topic;f=2;t=021739

Posted by Chris Bridges (Member # 1138) on :

Every time I hear statistics used in arguments, my first thought is: "How the heck do they know?"

Yes, we are assured that these polls are scientific and have a carefully calculated margin of error, but how can they be? How do you verify it?

I'm more likely to believe polls about relatively meaningless things. 58% of Americans prefer blue curtains? Sure, why not. But when I hear that 49% of Americans favor Bush's policies or 41% of Americans plan to vote for Kerry or 54% are pro-life or 4% are foot fetishists... I have to admit, I just ignore them.

Every poll on the Democratic candicacy this year has been completely useless in predicting what would actually happen. The media's helplessness about this amuses me no end.
In high school we were given various polls and surveys about our sex lives, our drug usage, whether or not we smoked or petted or drank or masturbated, etc, and I know for a fact that guaranteed anonymity answers or not, a lot of kids flat out lied.
Does anyone really think that surveys on adultery get honest responses?
I've heard that 10% of the population is homosexual. I've also heard the numbers 3%, 5%, 8%, 12%, etc, and the numbers tend to fluctuate depending on the organization presenting them. Has anyone here ever been asked on a survey if you were gay? What if you're gay but refuse to identify yourself? What if you're bi? What about people whose sexuality has changed since the question was asked? Is it good science to count 100 people and then multiply that out to a few billion and assume the percentages hold true?

Polls are also written by humans, who often curve the questions to get different answers, even if they don't realize they're doing it. Consider two different polls:
Do you support the liberation of Iraq?
Do you support the American invasion of Iraq?
A pollster might consider them the same question, but each version presupposes a certain attitude and more often than not will probably get it. The same person could vote yes to the first and no to the second with complete sincerity.

Then I hear that such-and-such poll was conducted on 2,000 people. This is supposed to reflect an entire country? The opinions of 2,000 people doesn't even reflect an entire city. What a poll reflects are the stated opinions of the people willing to be polled. That doesn't include the opinions of the people that didn't sign up for the poll or who walked away from the nice lady with the clipboard or hung up on the phone call, and it doesn't reflect the opinions of people who told the pollster what they thought they should believe instead of their honest opinion. Online polls are even worse, they're entirely self-selecting. Most polls even exclude whole classes of people: I work for a media organization, and just about every pollster that's ever called me has hung up after I mentioned it. So much for my opinion.

Unless you're dealing with the census, when everyone in the country is given the exact same questions and not given the choice to opt out, no percentage can accurately predict the majority until that percentage approaches the majority.

And as soon as the poll requires conscious human response, it's no longer scientific.

By the way, anyone interested in statistics must read "How to Lie With Statistics" by Darrell Huff. Written in 1954, and still excellent at playfully pointing out how anyone can skew data to fit their needs.

Posted by mackillian (Member # 586) on :

(I saw your title and read "Do you trust trolls?")

Posted by imogen (Member # 5485) on :

65% of people say they don't. And that's good enough for me.

Posted by sndrake (Member # 4941) on :

Maybe you should do put the question to a focus group and see what you come up with.

Hmmm...

Would a thread count as a focus group?

Posted by rivka (Member # 4859) on :

This group is self-selecting, both by responding to the thread and by being on Hatrack in the first place -- hardly random. [Wink]

OTOH:

quote:
Is it good science to count 100 people and then multiply that out to a few billion and assume the percentages hold true?

Yes. IF the group is selected randomly from the population being studied AND large enough (100 may or may not be, depending on a number of factors).

Posted by sndrake (Member # 4941) on :

Sheesh. Remind me to use smilies more often. [Smile]

Thought I was being clear enough that I was joking when I suggested a focus group to find out if people trust polls. [Dont Know]

I think maybe I'm all communicated out today. Way too much interaction with way too many of my fellow human beings. [Wink]

Posted by rivka (Member # 4859) on :

<-- has a dry wit, best served lightly chilled

I knowed you wuz kiddin'! Hence the [Wink]

!

Had I been serious, the [Wink]

would have been absent, and I would have elaborated. Probably would have linked to somewhere relevant.

Posted by pooka (Member # 5003) on :

No.

Posted by advice for robots (Member # 2544) on :

When I am polled, I make up more than half my answers on the spot. "Do you support Representative So-and-so's bill to defibrillate the Sanpete County sunlight/water ratio over the next 15 years?" "Sure."

Posted by Nick (Member # 4311) on :

Honestly, I don't trust polls. I'm not so sure on whatthenationthinks.com either. Plus they're very liberally biased. [Razz]

[ February 24, 2004, 12:39 AM: Message edited by: Nick ]

Posted by plaid (Member # 2393) on :

Chris -- my Dad worked for Gallup Poll and he agreed with a lot of what you said. He felt that Gallup's polls were pretty good (they'd had a lot of practice) and he talked a lot about how important it was to try to phrase questions as neutrally as possible -- he'd usually have to do serious rewriting of the questions that clients would want asked. A good news article describing a poll should always be careful to say what questions were asked and exactly how they were phrased.

One thing he did say was that when telephone polls happen, they're not reaching people who don't have phones or who've just changed services, etc. So polls don't reach homeless folks, college students, Mormon missionaries in Brazil, poor people who don't have phones, people like me who live on hippie communes, etc....

Posted by Bob_Scopatz (Member # 1227) on :

quote:
Do you trust polls?

Only Kama and the Pope.

Posted by Mr.Funny (Member # 4467) on :

Well, um, 89.763% of statistics are made up on the spot! [Wink]

Having taken a very very very basic course of statistics, I am relatively certain that (assuming the questions are non-biased, which they are, and assuming that people answer honestly) 2000 is a sufficient number of people as long as it is from a spectrum including most of the different population groups, like age, race, gender, etc. The number of people polled determines the +/- whatever percent. After a certain point, the +/- x% won't really matter a whole lot. Of course, this is all a moot point since the polls are almost always horribly skewed.

Someone should make a poll on Sakeriver asking whether or not you believe polls to be accurate.

Posted by Farmgirl (Member # 5567) on :

Chris,
We discussed this a lot in my college statistics class. So many of the popular polls never give any evidence of how much of the population was polled, how diverse the population was, etc. So most polls are crap, unless they want to release statistics about how they did their polling.

On-line polls -- what's to keep me from going in and voting over and over and over? That's hardly accurate.

Now, the bigger polling places that really do it officially, I believe them a bit more.

Farmgirl

Posted by Hobbes (Member # 433) on :

*Hearily agrees with Chris*

Hobbes [Smile]

Posted by Bokonon (Member # 480) on :

What, no love for me Bob?

Or is it because I'm only half a Poll, huh?

Huh?

-Bok

Posted by Bob_Scopatz (Member # 1227) on :

Bok...sorry to leave you out.

From now on, I'm going to call you by your new nickname: The Pollster!

[Big Grin]

Posted by sndrake (Member # 4941) on :

Bok,

I suspect that since you're "only half a Poll," Bob only half trusts you. [Wink]

Posted by jeniwren (Member # 2002) on :

Chris, I think what little trust I had in polls went out the window when all the polls "showed" that Dean was a serious contendah, and then seeing the results that actually came in from the election booths.

Posted by Teshi (Member # 5024) on :

quote:
Well, um, 89.763% of statistics are made up on the spot!

Thanks for stealing my unoriginal witty remark! [Smile]

Posted by Dan_raven (Member # 3383) on :

Once again, Bob stole my joke before I could post it.

How's this:

I only trust polls as far as I can throw them.

That means small fencing polls are a lot more trusted than those big telephone poles.

In defence of polls, in Iran, when a national polling company showed promblems under the beneficial Islamic Clerical government, that government immiediately moved to do something about it.

They closed the polling company and threatened to shoot the polsters if they tried anything like that again.