Why the Republicans Lost: Living in a Land of Make Believe
Posted: November 17, 2012 Filed under: 2012 elections | Tags: data, numbers geek, Polling, pollsters, scientific method 19 Comments“The eternal mystery of the world is its comprehensibility.”
Albert Einstein
There’s a really great piece in the NYT by economist Richard H. Thaler who explains that the really big winners were numbers geeks last week. I would also argue that the really big losers are the folks at FOX News, the Romneys of the world, and the religious and republican right who basically rely on old world views, religions, and reality denial. These are people who don’t rely on data. These are people that continually criticize intellectuals and folks that study the way things are. They are numbers deniers.
There is a limitation to forecasting things. You can’t predict the inevitable black swans, but you can identify trends, normality, and average. You can also–by systematically studying things–comprehend basic truths about the life, the universe, and eventually everything. Republicans have learned one small piece of this since they’ve decided on chasing the Hispanic vote. But, that’s a small take away compared to the big lesson. Most things are comprehensible if you drop the dogma, the sense of entitlement based on your frames, views based on ideology and your sense of intrinsic rightness. Those of us that work with data–not with wishful thinking and intransigent dogma–did win the day as Thaler suggests. A lot more Republicans would do well to learn the Law of Large numbers.
So it may come as a surprise that, collectively, polling companies did quite well during this election season. Although there was a small tendency for the pollsters to overestimate Mr. Romney’s share of the vote, a simple average of the polls in swing states produced a very accurate prediction of the Electoral College outcome. Notably, the most accurate polls tended to be done via the Internet, many by companies new to this field. That’s geek victory No. 1.
This relatively accurate polling data provided the raw material for the second group of election pioneers: poll analysts like Nate Silver, who writes the FiveThirtyEight blog for The New York Times, as well as Simon Jackman at Stanford, Sam Wang at Princeton and Drew Linzer at Emory University.
What do poll analysts do? They are like the meteorologists who forecast hurricanes. Data for meteorologists comes from satellites and other tracking stations; data for the poll analysts comes from polling companies. The analysts’ job is to take the often conflicting data from the polls and explain what it all means.
Worry about the reliability of the polling data led to widespread skepticism, or even outright hostility, toward poll analysts. The phrase “garbage in, garbage out” was one of the more polite criticisms bouncing around the Internet in the days before the election.
Because the polls were not, in fact, garbage, the first job of a poll analyst was quite easy: to average the results of the various polls, weighing more reliable and recent polls more heavily and correcting for known biases. (Some polls consistently project higher voter shares for one party or the other.)
Republicans live in a world of data denial. They cling to ‘trickle down” economics and the idea that taxing the “job creators” ruins economic growth even though decades of research show this to be untrue. Many deny the theory of evolution even though molecular biology and the ability to map genomes and identify the structure and particulars of DNA have pretty much made this theory as close to the iron clad truth as other science theories like gravity, magnetism, and Hawking Radiation. The same crowd denies climate change. The Republican base lives in a world of anti-intellectualism and continues to be left behind. It’s no wonder that they’re all freaked out about the election results. They embrace propaganda and superstition. They do not follow the data. They react like primitives who see fire or air planes for the first time.
Pundits making forecasts, some of whom had mocked the poll analysts, didn’t fare as well, and many failed miserably. George F. Will predicted that Mr. Romney would win 321 electoral votes, which turned out to be very close to President Obama’s actual total of 332. Jim Cramer from CNBC was nearly as wrong in the opposite direction, projecting that the president would win 440 electoral votes.
There is a lesson here. When it comes to assessing the chances of some complicated combination of events, gut feelings are pretty much useless. Pundits are no better at forecasting election outcomes than they would be at predicting the final path of a hurricane. Smart pundits should consider either abandoning this activity, or consulting with the geeks before rendering their guesses.
The deal is that that folks like Cramer and Will get face time on TV and print time in the press. This is too bad. Most numbers geeks live in a room with a database and a good stats program. They never get to meet the press or face the nation. Data mining and number krunchers helped the Obama team identified what was what on the way to the win.
The third set of folks who deserve recognition in this election cycle were a group of young people working in a windowless room at Obama headquarters, affectionately known as the cave. They were part of the effort by the numbers-oriented campaign manager, Jim Messina, to maximize turnout.
THERE are two basic parts of an election campaign. The first comes under the category of messaging — deciding what a candidate should say and what ads to run. Most of the commentary we read about elections focuses on this component.
The second part is turnout, and in some ways is even more important. Here is a simple bit of math that you don’t have to be a geek to understand: It doesn’t matter which candidate a person prefers unless that person shows up and votes.
Pundits will debate for eternity which campaign did a better job of communicating its message, but there is no doubt which campaign won the turnout contest. Young, black and Hispanic voters all turned out in higher numbers than expected, and they often supported President Obama.
Much was made of the big Obama advantage in field offices in swing states. But those field offices would have been little good to the campaign without modern tools to find potential voters, have them register and encourage them to vote. In the weeks leading up to the election, the Obama canvassers had accurate lists of potential voters and field-tested scripts for their contacts with voters. This explains in part why Democrats were such heavy users of early voting.
I’ve spent my life in a land of data and I can tell you that I’ve told quite a few clueless CEOs like Romney that their view of their business is wrong and unsupported by the numbers. I’ve been in two corporations where the senior management was making bad decisions on gut feelings and wishful thinking. I came in with data, showed them what was what, and that they were basically running the company into the ground and that their companies were going bankrupt. In both these cases, bankruptcy happened. They looked at reality too late for it to be of any use. I’ve also done research that’s been passed over–later to be proven true–simply because folks don’t want to believe that banks would be so stupid as to systematically give increasing numbers of bad loans. People deserve information. Republicans tend to spew propaganda and sermons.
Numbers denial runs strong with folks that would rather believe what they want to believe than look at patterns, trends, and information that would be right under their noses if they’d only allow it. What I’m hoping–more than anything else–is that this election shows how dangerous magical thinking can be. I’m not too hopeful because human history is littered with bad, destructive magical thinking.
The earth is flat. The earth is the center of the universe. The earth is 8000 years old.
People that embrace magical thinking should not be making decisions for the rest of us. That should really be the take away from this election. There are still people leading the Republican party that embrace the idea of Dinosaurs living in the Garden of Eden. These people are working on the way they deliver the message but they are not changing their actual beliefs.
A prime example of this is Gov Bobby Jindal whose 2016 campaign for the president is on full throttle. He’s already calling for immigration reform. He’s called for the Republican Party to stop being the “stupid party”. Yet, look at his record as Louisiana Governor. His education reform initiative includes teaching creationism and draining public funds for private schools that will have no education requirements or accountability. He pushed through some of the harshest measures restricting women’s access to reproductive health care. He has refused to implement necessary health care reforms and has turned down funds that would help the state’s many poor. He has been selling state assets--including hospitals and jails–to private corporations. He’s earned the name Dr. Destructo here. He’s also well known for his college writing on exorcism. He’s interfered in Iowa politics by supporting groups that want to take down a judge because of his findings on gay marriage. The man is a walking nut job with endless ambition and ruthlessness.
Jindal’s got the Republican mentality of reworking the message while still doing the crazy stuff down pat. The Republican party and Republican Leaders like Bobby Jindal believe that they really don’t have to drop the crazed, magical thinking for reality and data. Jindal just believes in delivering the right message and the opposite policy. Until the Republican party reforms its core values, voters will have to watch their actions. Again, that’s a form of data gathering isn’t it? You can’t deny there’s been a war on women if you look at the number of anti-women laws that have come up at the national or state level. You can’t deny there’s an anti-science bias in the party when you actually look at the number of things they fund and defund at both the national and state level. They all really need to just wake up and look at the data for a change. After all, that’s really why they lost the election.
Nate’s Numbers: the numbers converge
Posted: November 5, 2012 Filed under: 2012 elections, open thread, polling | Tags: Nate Silver, Number Krunching, open thread, Polling, polls, Punditry, Sam Wang, Statistics 60 Comments
I’ve spent all my professional life drenched in numbers and statistics so Nate Silver’s numbers fascinate me. It’s probably the same reason they drive Republicans and pundits to distraction. Unraveling trend is easier with numbers than hateful, wishful thinking motivated by political piety. So, Karl Rove got on TV–probably trying to save what’s left of his credibility–saying that Romentum was stopped by Sandy. Romentum was a bit of canard and it turns out so is Sandy. Silver tries to discern the possible factors behind the recent numbers and looks at the Sandy Factor. That’s a relatively simple task for any one with a database and a background in trend analysis.
When the hurricane made landfall in New Jersey on Oct. 29, Mr. Obama’s chances of winning re-election were 73 percent in the FiveThirtyEight forecast. Since then, his chances have risen to 86 percent, close to his highs on the year.
But, while the storm and the response to it may account for some of Mr. Obama’s gains, it assuredly does not reflect the whole of the story.
Mr. Obama had already been rebounding in the polls, slowly but steadily, from his lows in early October — in contrast to a common narrative in the news media that contended, without much evidence, that Mr. Romney still had the momentum in the race.
Moreover, there are any number of alternatives to explain Mr. Obama’s gains before and after the storm hit.
- Mr. Obama was adjudicated the winner of the second and third presidential debates in surveys of voters who watched them.
- The past month has brought a series of encouraging economic news, including strong jobs reports in October and last Friday.
- The bounce in the polls that Mr. Romney received after the Denver debate may have been destined to fade in part, as polling bounces often do following political events like national conventions.
- Democrats have an edge in early voting based on states that provide hard data about which party’s voters have turned out to cast ballots. Some voters who were originally rejected by the likely voter models that surveys apply may now be included if they say that they have already voted.
- Both Mr. Obama and Mr. Romney have been running lots of advertisements, which could have some effect, especially in the swing states.
- Mr. Obama’s voter-targeting operation may in fact be stronger than Mr. Romney’s and may have begun to show up in the polls.
- Mr. Obama’s approval rating is at 49 or 50 percent in many surveys, a threshold that would ordinarily predict a narrow re-election for an incumbent.
- Some elections “break” toward one or another candidate at the end as undecided voters tune in and begin to evaluate their decision.
Each of these hypotheses could merit its own article. But the point is that the causes for Mr. Obama’s gain in the polls are overdetermined, meaning that there are lot of variables that might have contributed to the one result.
If I had told you in January that Mr. Obama’s approval rating would have risen close to 50 percent by November, and that the unemployment rate would have dropped below 8 percent, you likely would have inferred that Mr. Obama was a favorite for re-election, with or without a hurricane and what was judged to be a strong response to it.
Whatever the causal factors, Nate’s numbers look good for the President. Sam Wang–a Princeton number kruncher–says it not only looks like the President will hold his office but that the US Senate might see a Democratic Pick up of two. This is an important firewall for those of us that care about things like Supreme Court appointments and getting rid of the filibuster silliness that has allowed the Republicans to basically thwart governing. Dems may pick up Nebraska and Pennsylvania.
Rather than responding to the analysis, the Republicans continue to attack Nate Silver the man. You may recall this Krugman piece last week about The War on Objectivity.
Brad DeLong points me to this National Review attack on Nate Silver, which I think of as illustrating an important aspect of what’s really happening in America.
For those new to this, Nate is a sports statistician turned political statistician, who has been maintaining a model that takes lots and lots of polling data — most of it at the state level, which is where the presidency gets decided — and converts it into election odds. Like others doing similar exercises — Drew Linzer, Sam Wang, and Pollster — Nate’s model continued to show an Obama edge even after Denver, and has shown that edge widening over the past couple of weeks.
This could be wrong, obviously. And we’ll find out on Election Day. But the methodology has been very clear, and all the election modelers have been faithful to their models, letting the numbers fall where they may.
Yet the right — and we’re not talking about the fringe here, we’re talking about mainstream commentators and publications — has been screaming “bias”! They know, just know, that Nate must be cooking the books. How do they know this? Well, his results look good for Obama, so it must be a cheat. Never mind the fact that Nate tells us all exactly how he does it, and that he hasn’t changed the formula at all.
This is, of course, reminiscent of the attack on the Bureau of Labor Statistics — not to mention the attacks on climate science and much more. On the right, apparently, there is no such thing as an objective calculation. Everything must have a political motive.
This is really scary. It means that if these people triumph, science — or any kind of scholarship — will become impossible. Everything must pass a political test; if it isn’t what the right wants to hear, the messenger is subjected to a smear campaign.
Any kind of scholarship has become challenging under Republican fanaticism as witnessed by the attacks on evolution, climate change, and the economic analysis that shows there is no such thing as an economic benefit created by low marginal tax rates for the rich. Just ask scientists trying to get grants to study things like stem cell research. Fox gets people to believe anything. Science causes them to retreat to their medieval churches and scream about intervention by celestial beings. (So, if gawd caused Sandy to take out NJ and NY because of Gay Rights, does this mean gawd caused Sandy to give Obama momentum? Ask Grand Inquisitor Pat Robertson about that one.)
Yes, Nate’s numbers took another upward tick this morning. He also believes that Obama will take the popular vote which should be dismaying to all those journalists that keep wanting to turn this into a white knuckle election.
Silver also added to Obama’s likely number of electoral votes on Monday. He now sees the president winning 307.2 to 230.8 for Mitt Romney, a tiny tick higher than he saw the race on Sunday.
He also sees Obama capturing the popular vote, taking 50.6 percent to Romney’s 48.5.
Rachel Maddow insists the Republicans see the numbers and actually believe that their man Mittster is a goner. The blame game has already begun. Haley Barbor blames Sandy. Lindsey Graham is actually looking at numbers and notices that that demographics are not in their favor.
QUOTE OF THE DAY: ”If I hear anybody say it was because Romney wasn’t conservative enough I’m going to go nuts. We’re not losing 95% of African-Americans and two-thirds of Hispanics and voters under 30 because we’re not being hard-ass enough.”
– Politico quotes Sen. Lindsey Graham (R-SC), saying that demographics would be the only reason for a hypothetical Mitt Romney loss Tuesday.
Yeah, the blame game has begun and the election isn’t even over. Funny he should bring up Hispanic and black voters. Latino Decisions found that Hispanics support President Obama in historic numbers — 73 percent. They believe that’s enough to carry President Obama to victory in four swing states and ultimately to win re-election. Those four swing states are Nevada, Colorado, Virginia, and Florida.
This is closer to the truth. Republican policy statements as expressed by “severely conservative” Mitt have driven off the young, hispanics, blacks, and women. I had MSNBC on mute most of the weekend when they were following the candidates around. All you have to do is look at the people used as back drops for candidates’ speeches to realize who really is on the losing side of US demographics. This is part of the numbers game. You can’t build momentum or fantasize a trend based on capturing an ever decreasing slice of the American Pie.
Which brings me to the post election deconstruction that should occur in the Republican party. When will the actually give up on the Southern Strategy? We’ve seen
more race baiting in this election that I’ve frequently wondered if the ghost of George Wallace is running the Romney campaign. We’ve seen attacks on women’s rights that make me wonder if Republicans know that women got the right to vote. We’ve seen support of policies that are based on show us your papers that remind me of old NAZI movies. You can’t attack and demonize the majority of the electorate and expect the numbers to come in for you. You also can’t build policy on attacking scientific models and theories. The Republican party has definitely shown that its plan for America is to try to recreate the past no matter what the cost.
All I can say is go Nate go! Win one for Ada Lovelace and the country. There are easy ways to figure out which events contribute to trend and what’s just random. People should pay more attention to Nate Silver and a lot less attention to the likes of Karl Rove and Haley Barbour. Nate Silver bet Joe Scarborough $2000 that his analysis was right saying “Occam’s Razor: Pundits are useless”. That’s something I completely grok.
“I think I get a lot of grief because I frustrate narratives that are told by pundits and journalists that don’t have a lot of grounding in objective reality,”
The Selective Mutism of the Progressive Village
Posted: February 7, 2011 Filed under: Egypt, income inequality, POTUS | Tags: 2011: days of revolt, Obama Hearts Reagan, Polling, progressive punditry class, Working and Middle Class Unite 32 CommentsThis is going to be long, but it won’t work well to separate it into two posts. So I’ve divided it up into sections that you can read if it’s too much to digest in one sitting.
Part I: Obama’s Truly-Significant-Best-Month-Ever is O-V-E-R
Exhibit A
Huffington Post/Chris Weignant, February 2nd:
In January, President Obama’s approval rating went significantly higher, while his disapproval rating continued a trend of dropping with a big spike downward. What both of these meant, taken together, is that Obama is once again “above water” in the polls, with his approval rating beating his disapproval rating. This hasn’t happened since last June. But, in reality, Obama has pretty much erased his past entire year’s slow slide in poll numbers — in a single month. Obviously, he didn’t hit an all-time high in absolute numbers, but still, when taken month-to-month, January, 2011, was Obama’s best month of his entire presidency. Not only did he finally get his bump — but it was a truly significant bump.
Exhibit B
Gallup Daily, February 2nd-4th:
Gallup Daily, February 3rd-5th:
I’m not going to waste time putting the Rasmussen tracking chart up, but it shows the same drop. Steve M. over at No More Mister Nice Blog has this to say about it:
This Rasmussen poll has gotten a bit of attention, and not exclusively from the right
Okay, let me interject there for a moment, because I think Obama’s poll numbers dropping off has gotten attention “exclusively from the right”–or at least in terms of what people will cop to paying attention to openly. Since I haven’t seen much discussion in the progressive Village making itself readily available, I’ve been combing through blogs and news outlets trying to find any commentary on the complete reversal of the hyperbolic narrative that was floating just a few days ago–that Obama was King of the Polls again–but almost all of the discussion I’m seeing of Obama’s approvals tanking is coming from the usual wingnut suspects. If you click on Steve M’s “a bit of attention” link, you’ll see an archive of the memeorandum listings under the item on the Rasmussen polling numbers: James Joyner, Gateway Pundit, Hot Air, Scared Monkeys… (the Jennifer Rubin link there doesn’t even discuss the Rasmussen poll.) I don’t see any lefty or even moderate names there, do you?
Anyhow, Steve M continues:
Yeah, yeah, it’s Rasmussen — though, as James Joyner notes, the numbers have worsed in the new Rasmussen poll compared to old Rasmussen polls. Presumably the right-wing bias hasn’t worsened, right? (Call me naive, but I don’t think Rasmussen just makes these numbers up — I think the polls have a right-wing sample bias, and the bias is baked into the data, but that there’s real polling going on nonetheless.)
The reason I take this somewhat seriously is that similar things seem to be happening in Gallup’s daily Obama approval tracking poll — run your cursor over the graph and you see that the president’s approval number was solidly ahead of his disapproval number for much of late January, peaking at 50%-41% in the January 27-29 period. Now it’s down to 45%-47%.
Or, rather, it’s back down to 45%-47%. That’s roughly where Obama was in the Gallup poll pretty consistently from June through early January.
His rationale is that “Obama approval has just returned to baseline”:
So I don’t think Obama’s being hurt by his response to the situation in Egypt (a meme the right would desperately like to spread) so much as he’s not being helped anymore by the three things that met with public favor in the past month and a half or so — the productive lame duck session, the State of the Union address, and (especially) the very well-received Tucson speech.
Wait just a frick-on-a-stickin’ minute there…
Did Steve M just include the president’s SOTU address as one of the three things that met with public favor and had helped his ratings? I’m not so sure about that. In fact, I think it was such a lackluster and forgettable speech that the after-effects of what was left out of the speech damaged his credibility. As Charles Blow noted in response to Obama’s annual address:
President Obama made history on Tuesday.
It was only the second time since Harry S. Truman’s State of the Union address in 1948 that such a speech by a Democratic president did not include a single mention of poverty or the plight of the poor.
And, that’s not all Obama left out. While revolution was erupting in Egypt, with its middle and working class citizens joining together and rising up to demand their human rights and–among other things–an end to persistent unemployment, the president of the United States uttered the words “Egypt” and “Egyptians” not once.
I don’t think in light of what has happened over the last week that Obama’s speech served him well at all. Sure, various instant analysis polls afterward were inflated with happy campers, but that’s out of the people who thought it was important enough to watch the speech in the first place. If you go by the Nielsen numbers, there’s a drop off there too… for goodness sake, even Perez Hilton kept track:
Less people were interested in what President Obama had to say this year.
About 43 million people watched his State of the Union address Tuesday night, which was down in viewership from the previous year. In fact, about 11% less people watched the speech.
There was also No SOTU Bump for Obama this year.
I think once Americans had a chance to sit back, forget the words that were in the speech, and observe the events that transpired in their wake, the words that were missing from the president’s address (poor, poverty, Egypt, Egyptians…) have come into stark and stunning relief. Obama is not a “different” kind of politician or president–he is an indifferent one.
If you’re reading this on the frontpage and are interested, there’s a Part II, III, and IV after the fold.









Recent Comments