Subscribe to Desicritics
Why do Exit Polls Fail?
Chandra
Haven’t you often wondered about why exit polls fail to forecast in India? Here is an ‘inside’ insight into why they fail.
One of the biggest myths about exit poll failures is the sample size. Often, I hear on TV (and cringe) about how 300000 exit poll participants can forecast who 300 million voters would for. Statistical theory says you could forecast 300 million voting intentions with just 50000 exit poll participants with almost no error. Of course as the number goes up the error comes down. But that’s theory for you. In reality, sample sizes are dependent on the degree of homogeneity within the voting population. India, we know is nowhere near homogenous. In other words, higher or lower sample sizes will not alone solve the problem.
Coming to the first reason for exit poll failures, it is not how many we interview about who they voted, it is where we we slected them from that is critical. In other words, is a voter from Kalkaji in South Delhi a better representation of voters in South Delhi than a voter from Greater Kailash-II? Is a voter from Koramangala a better representation of voters in Kanakpura Lok Sabha seat than a voter from Madiwala? (Sorry readers who are not familiar with these two locations, I could go on and on with other city examples). It is these challenges that pollsters often grapple with and that determines success or failure. The typical trend is for the under-representation of poorer voters in these surveys and often parties who get these votes tend to be under-represented in the final seat share forecasts.
The second big problem is-the lower the difference in real vote share between the political parties, the more difficult it is to forecast. In 1998, the difference in votes between the BJP and Congress in Madhya Pradesh was 1.85% votes, well within the margin of error. No wonder everybody got it wrong. In recent elections, most pollsters are reluctant to forecast if there are too many seats within the margin of error.
The third problem is – the quality of data collection. Corruption is endemic in Indian society (not just Government). Many data collectors are freelancers (college students’ et al) and some of them resort to fudging data (a common problem in the Indian polling industry). This pollutes the incoming data and the subsequent analysis. Unlike corporate research which has a good system of back checking and other quality checks, it is impossible to back check a voter’s voting choices.
The last problem is – the impact of voter turnout. It is often possible that polling booths with reliable patterns suddenly show a spurt in voter interest, usually for a single party. This could totally skew the exit poll results. For example, a polling booth could see a massive increase in previously disenfranchised Dalit voters. Now if that booth were a part of the exit poll survey, we would end up with a higher forecast for the BSP. On the other hand, if that booth were not a part of the exit poll survey, we would end up placing a low vote share for the BSP. Current models in India are unable to model accurately changing voter turnout.
This being the situation, I am strongly in favour of using opinion polls to manage elections. Of course it can and will be only one component of your overall election strategy, albeit a critical one.
As far as exit polling is concerned, the future is really in pooled sampling. With more and more elections likely to be within the margin of error, increased coverage is the way forward. That would mean a large common data collection pool covering more polling booths and constituencies but separate forecasts by individual agencies. Is somebody listening?
Why do Exit Polls Fail?
RSS:
- Subscribe to RSS 2.0 feeds for:
- » Comments on this article
- » Politics
- » Politics: Elections
- » Politics: India
- » Desicritics.org articles by Chandra
- » All Opinion articles
- » All Desicritics.org articles
Ritika Sharma
July 1, 2007
06:42 AM
WHAT A LOAD OF CRAP..............
AnArch
July 1, 2007
07:06 AM
I think you've said that already - anything productive to add?
Ritika Sharma
July 2, 2007
05:14 PM
well sweety....judging by the number of comments on your post i would have to say ...most people found it......a load of crap.
Ritika Sharma
July 3, 2007
11:28 AM
Just love watching your crap....sink without even creating a ripple....
Chandra
May 27, 2008
01:16 AM
I would caution readers to track CNN IBN exit Polls with caution. Somehow i am beginning to detect a left wing bias in the analysis. Dorab has been closer to the actual numbers.
Kerty
May 27, 2008
02:57 PM
Chandra..
I think you nailed it.
And the same is true of any kind of polling or statistics in India. It is just impossible to be accurate or meaningful.
commonsense
May 27, 2008
06:09 PM
hey kerty, you're BACK!!!! How's your Dosti pub doing? are you all set to post long, barely readable messages for all of us? look forward to tbem...
Chandra
May 28, 2008
09:12 AM
Kerty Sahab
Kahan gayab ho gaye aap?
kerty
May 28, 2008
10:06 AM
Travels, business and social/family events take me out for extended period. Plus too much blogging can become addictive and yet get boring, repetitive - so it feels good to take a break from it.
kerty
May 29, 2008
09:20 PM
I do not think polls in India are conducted to assess the real mood of electorate or assess local ground realities. Therefore, polls remain merely a packaged political product. They are used primarily to influence the undecided voters at large, and to spin certain political points that fit within self-serving political strategies.
Swapan Dasgupta has a nice write up on polls
http://www.dailypioneer.com/columnist1.asp?main_variable=Columnist&file_name=swapan%2Fswapan200%2Etxt&writer=swapan
Chandra
May 30, 2008
12:37 AM
Kerty
Please have some degree of respect to other people's professions. Most pollsters I know (and I know many) are sincere, unbiased hardworking folks. Please donot say what you said again.
Chandra
May 30, 2008
12:39 AM
Kerty
Swapan dasgupta can shut up. Will he offer to quit if any of his journalistic comments turn out to be untrue? He has made many off late. The poor guy is stuck between trying to cosy upto MM Singh and BJP at the same time...
Kerty
May 30, 2008
01:48 AM
Chandra
Just because something is called a profession does not mean it has evolved integrity or that it must be respected.
There is a difference between an an op-ed commentator and a pollster. The former opines about what he thinks and he does not have to be right, correct, unbiased, non-partisan in his/her commentary. The pollster on the other hand is posing to 'report' and 'analyse' what other people think and present it as something factual, scientific, statistical - therefore, his/her reporting requires integrity. We expect reporters to be factual, accurate, unbiased, non-partisan. And when they are, they would get the respect they deserve. Just because someone is eking a living at it does not earn them respect automatically. I do not have to respect hookers just because it is their profession and they work hard at it.
Chandra
May 30, 2008
07:56 AM
Kerty
I have not read a more mindless comment as the one above.
commonsense
May 30, 2008
09:39 AM
chandra,
you must be new to kerty....now you will figure out what i meant in post #7 above...
kerty
May 30, 2008
08:01 PM
Chandra
Care to elaborate how/why some of the arguments presented by me or Swapan are devoid of relevance or logic?
commonsense
May 30, 2008
08:59 PM
Chandra,
Care to pick up Kerty's bait? Care to explain precisely why illogical arguments are, ummm, illogical?
Chandra
May 31, 2008
03:22 AM
Kerty-16
Because of none of your judgements and pronouncements about polling are based on facts. And if you think they are....present the facts....List all the states that went to elections in the last 4 years, list all the polls carried out in these states and indicate the variance from the result.....Then, compare this with another country with a large number of parties. Compare the variance in both countries and see if the levels of accuracy are different.
Kerty
May 31, 2008
10:36 AM
Chandra..
Given the diametrically diverse and pluralistic conditions in India, no amount of statistical extrapolation can ever be accurate in India. That is why democracies have periodic elections to measure voter sentiments. Than what is the complusions to have polls? Who really needs them? Only those who want to manipulate voters. Its primary purpose is to manipulate voters. That is why I equated polls to hookers - they try to be a substitute/compliment for the real thing but it can never be real thing. Polls can never be accurate in India in assessing what voters think and want. Pollsters can only be as accurate as astrology - its like dozens of astrologers predicting all likely scenarios and when one of the scenario comes out to be true, you would have at least one astrologer who would be right in predicting it and he than becomes a instant celebrity and touted for his impeccable credentials for accuracy. I can show you such astrologers if you can show me pollsters who have been accurate.
Add your comment
(Or ping: http://desicritics.org/tb/5663)