Keep on spreadin’ that propaganda
Iraq, National Politics Comments (4)
Well folks I just flat out missed The Lancet‘s cluster survey (registration required) of Iraqi civilian deaths.
Others, however, have not.
Stageleft, for instance, quotes them at face value (and to do him justice, he does say that the numbers are higher than he would have guessed). The information he used came from a CBC story.
Now, buried deep within that CBC story is this interesting tidbit: “The authors acknowledge that the data cited in the study might be of ‘limited precision.’”
Indeed? Limited precision? Say it ain’t so!
Look up the summary interpretation. It says “We estimate that 98 000 more deaths than expected (8000–194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included.” According to Fred Kaplan over at Slate this formerly read “We estimate that 98 000 more deaths than expected (95% CI 8000–194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included.” The difference is that the version I saw at The Lancet doesn’t mention the confidence interval (the “95% CI” part).
So, what does that mean? I’ll let Mr. Kaplan explain in his own special way:
Readers who are accustomed to perusing statistical documents know what the set of numbers in the parentheses means. For the other 99.9 percent of you, I’ll spell it out in plain English—which, disturbingly, the study never does. It means that the authors are 95 percent confident that the war-caused deaths totaled some number between 8,000 and 194,000. (The number cited in plain language—98,000—is roughly at the halfway point in this absurdly vast range.)
This isn’t an estimate. It’s a dart board.
Imagine reading a poll reporting that George W. Bush will win somewhere between 4 percent and 96 percent of the votes in this Tuesday’s election.
He’s right, you know. Back in my collegiate days when I was a budding political scientist I would have been raked over the coals (and justly so) had I turned in a paper with such stupid findings.
Now I’m not faulting Stageleft for not going to The Lancet or for not getting his news from the sources I use (hat tip on this one goes to MedPundit [who is quickly becoming a favorite read for me]). Indeed, it’s not his fault that the CBC doesn’t include this vital information. In fact, the CBC version isn’t the worst I’ve seen as the BBC does an even worse job of it.
The fault here really lies in two places:
- News organizations that are eager to print information less than flattering about the situation in Iraq. Imagine what the headline would have to read if they did the actual article justice: “War has killed between 8,000 and 194,000 Iraqis: study”
- The article’s authors for pumping up the middle-of-the-road guesstimate. As Kaplan explains:
Gilbert Burnham, one of the co-authors, told the International Herald Tribune (for a story reprinted in today’s New York Times), “We’re quite sure that the estimate of 100,000 is a conservative estimate.” Yet the text of the study reveals this is simply untrue. Burnham should have said, “We’re not quite sure what our estimate means. Assuming our model is accurate, the actual death toll might be 100,000, or it might be somewhere between 92,000 lower and 94,000 higher than that number.”
Does Dr. Burnham bear some responsibility for people taking that guess as being factual? If anything he’s now given the headline writers an out for their hype of the same figure. I’d say he certainly does.
So, what’s the significance here? There’s probably none. As Tim Worstall points out at Tech Central Station:
Look at the relative risk ratios (leave out Falluja; I don’t think anyone is really very surprised to see a higher mortality rate there): 1.1-2.3. It isn’t just that it is an absurdly wide one (note, a relative risk ratio of 1 would mean no effect whatsoever) it is that if this paper was written to generally accepted statistical standards it would never have been published. With a 95% confidence level a relative risk ratio of anything less than three is regarded as statistically insignificant. Just to clarify that, by “insignificant” no one is stating that it is not important to those people who undoubtedly have been killed during the War. What is being said is that we don’t have enough information to be able to say anything meaningful about it. “Statistically insignificant” means “we don’t know”.
MickC @ October 30, 2004