Remain Calm …Posted: October 29, 2012
The latest republican trick is one again to attack the polls, lie about the polls, and make up shit. Here’s some great pollsplaining from one of the great nerds of statistics and polls Sam Wang.
First, thing up … there is no Ro-mentum and Obama’s lead is static.
In either case, the overall picture is the same: a narrow Obama lead that is static – or perhaps widening. There is no evidence for Ro-mentum.
I’ve been pulling my hair out over the total lack of understanding by a lot of media punditry over white noise or random variation. I think it’s ignorance and not deliberate. Any series of numbers reported over time will go up or down randomly because that’s just the behavior of a variable reported over time. That’s when you have to apply the idea of what’s the top and bottom border of are representing random. This is measured by Standard Deviation and it’s what underlies the idea of being within the margin of error. You have to see fairly clear and sustained patterns in order to call it not random. That’s why you can’t hang on every up and down. Basically, it takes 7 ups or 7 downs to bust out of the three standard deviation realm of what’s normal. Take a look at the graph over there. It goes up and it goes down. The deal to figure out is what is ‘statistically’ significant. You can’t hang on every blip of every poll.
Poll correctness has a lot to do with the way the sample is treated. That’s the basis of the guess of how many republicans you should sample or democrats in the overall sample. You have to estimate–base on historical patterns or something else–what the patterns of voter turn out will be. That’s what most of the complaints about polls tend to reflect. These assumptions lead to weighting the samples differently. That’s why a lot of folks do meta-analysis or look at a poll of polls. This has to do with the law of large numbers which tells us that averages of averages tend to reflect reality. But, the core reality is that pollsters make money by producing accurate pictures and not by telling people what the want to hear. One legitimate concern is if you should look at all voters or likely voters. A lot of the rest of the so-called sampling biases that pundits scream about tend to be based more on wishful thinking than anything. There are some differences in terms of how statistics are used. Both Gallup an Rasmussen are median-based polls and this can make a difference when comparing to a poll that’s not based on the same metrics.
Wang points to the echo chamber and the Romney minions that are focusing on single polls and screaming or clapping.
Paul Krugman is calling out National Review Online for their attempted takedown of Nate Silver for biased methods and somehow cooking the books. Krugman writes:
This is, of course, reminiscent of the attack on the Bureau of Labor Statistics — not to mention the attacks on climate science and much more. On the right, apparently, there is no such thing as an objective calculation. Everything must have a political motive.
Now more commentators on the right, including Jay Cost (The Weekly Standard) and Jennifer Rubin (Washington Post), are getting in on the act. Wow, dogpile on the rabbit!
A popular approach to undermining technical knowledge is to throw mud, assert expertise, make picky points, and sow doubts among the less savvy. In this case, what’s the argument? The NRO writer, Josh Jordan, makes this core criticism:
When you weight a poll based on what you think of the pollster and the results and not based on what is actually inside the poll (party sampling, changes in favorability, job approval, etc), it can make for forecasts that mirror what you hope will happen rather than what’s most likely to happen.
Jordan sounds like many partisan polling enthusiasts – on both sides. However, his style of poll-dissection can very easily lead a person astray. The human mind has a large capacity for finding reasons to reject a piece of disagreeable evidence. I’ve written about this in the context of how people form false beliefs in politics (“Your Brain Lies To You,” NYT, June 27, 2008). Polling internals lend themselves very well to such “motivated reasoning.” It is always possible to find something not to like in a poll. This is why I discourage all of you from chewing over single polls.
Part of the reason that conservatives are hating on Nate Silver is that they aren’t getting the results they want. It’s the idea that if you scream loud enough, you can get things to change. Check out Silver’s latest. Romney’s chance at getting the electoral college is less than 30% which is basically where it’s been for a very long time . A few up blips do not change a long term trend unless they occur over a period of time.
Any way, calm down and let the metrics be with you. Here’s some interesting snapshots of trends on a state by state basis.