Stick A Second Lancet In It

UPDATE: below

The two Lancet studies estimating post-invasion Iraqi deaths were wildly out-of-kilter with other tallies--indeed, some said, "possibly fraudulent"--and rarely considered reliable. Except, that is, among the liberal MSM and lefties--in other words, says AllahPundit, by "every Democratic politician and nutroots blog in Nutrootsville."

Whatever credibility this politically motivated "bogus study" retained was dashed by a lengthy essay in Friday's National Journal:

[T]he authors have declined to provide the surveyors' reports and forms that might bolster confidence in their findings. Customary scientific practice holds that an experiment must be transparent -- and repeatable -- to win credence. Submitting to that scientific method, the authors would make the unvarnished data available for inspection by other researchers. Because they did not do this, citing concerns about the security of the questioners and respondents, critics have raised the most basic question about this research: Was it verifiably undertaken as described in the two Lancet articles?

"The authors refuse to provide anyone with the underlying data," said David Kane, a statistician and a fellow at the Institute for Quantitative Social Statistics at Harvard University. Some critics have wondered whether the Iraqi researchers engaged in a practice known as "curb-stoning," sitting on a curb and filling out the forms to reach a desired result. Another possibility is that the teams went primarily into neighborhoods controlled by anti-American militias and were steered to homes that would provide information about the "crimes" committed by the Americans.

Fritz Scheuren, vice president for statistics at the National Opinion Research Center and a past president of the American Statistical Association, said, "They failed to do any of the [routine] things to prevent fabrication." The weakest part of the Lancet surveys is their reliance on an unsupervised Iraqi survey team, contended Scheuren, who has recently trained survey workers in Iraq. . .

With the original data unavailable, other scholars cannot verify the findings, a key test of scientific rigor.

• Response rate. The surveyors said that 1.7 percent of households -- fewer than one in 50 -- were unoccupied or uncooperative, even though questioners visited each house only once on one day; that answers were taken only from the household's husband or wife, not from in-laws or adult children; and that householders had reason to fear that their participation would expose them to threats from armed groups.

To Kane, the study's reported response rate of more than 98 percent "makes no sense," if only because many male heads of households would be at work or elsewhere during the day and Iraqi women would likely refuse to participate. On the other hand, Kieran J. Healy, a sociologist at the University of Arizona, found that in four previous unrelated surveys, the polling response in Iraq was typically in the 90 percent range.

The Lancet II questioners had enough time to accomplish the surveys properly, Burnham said.

• Lack of supporting data. The survey teams failed to collect the fraud-preventing demographic data that pollsters routinely gather. For example, D3 Systems, a polling firm based in Vienna, Va., that has begun working in Iraq, tries to prevent chicanery among its 100-plus Iraqi surveyors by requiring them to ask respondents for such basic demographic data as ages and birthdates. This anti-fraud measure works because particular numbers tend to appear more often in surveys based on fake interviews and data -- or "curb-stoning -- than they would in truly random surveys, said Matthew Warshaw, the Iraq director for D3. Curb-stoning surveyors might report the ages of many people to be 30 or 40, for example, rather than 32 or 38. This type of fabrication is called "data-heaping," Warshaw said, because once the data are transferred to spreadsheets, managers can easily see the heaps of faked numbers.

• Death certificates. The survey teams said they confirmed most deaths by examining government-issued death certificates, but they took no photographs of those certificates. "Confirmation of deaths through death certificates is a linchpin for their story," Spagat told NJ. "But they didn't record (or won't provide) information about these death certificates that would make them traceable."

Under pressure from critics, the authors did release a disk of the surveyors' collated data, including tables showing how often the survey teams said they requested to see, and saw, the death certificates. But those tables are suspicious, in part, because they show data-heaping, critics said. For example, the database reveals that 22 death certificates for victims of violence and 23 certificates for other deaths were declared by surveyors and households to be missing or lost. That similarity looks reasonable, but Spagat noticed that the 23 missing certificates for nonviolent deaths were distributed throughout eight of the 16 surveyed provinces, while all 22 missing certificates for violent deaths were inexplicably heaped in the single province of Nineveh. That means the surveyors reported zero missing or lost certificates for 180 violent deaths in 15 provinces outside Nineveh. The odds against such perfection are at least 10,000 to 1, Spagat told NJ. Also, surveyors recorded another 70 violent deaths and 13 nonviolent deaths without explaining the presence or absence of certificates in the database. In a subsequent MIT lecture, Burnham said that the surveyors sometimes forgot to ask for the certificates. . .

• Follow the money. Virtually everyone connected with the study has been an outspoken opponent of U.S. actions in Iraq. (So are several of the study's biggest critics, such as Iraq Body Count.) Whether this affected the authors' scientific judgments and led them to turn a blind eye to flaws is up for debate. . .

Lancet II was commissioned and financed by Tirman, the executive director of the Center for International Studies at MIT. (His most recent book is 100 Ways America Is Screwing Up the World.) After Lancet I was published, Tirman commissioned Burnham to do the second study, and sent him $50,000. When asked where Tirman got the money, Burnham told NJ: "I have no idea."

In fact, the funding came from the Open Society Institute created by Soros, a top Democratic donor, and from three other foundations, according to Tirman. The money was channeled through Tirman's Persian Gulf Initiative. Soros's group gave $46,000, and the Samuel Rubin Foundation gave $5,000. An anonymous donor, and another donor whose identity he does not know, provided the balance, Tirman said. The Lancet II study cost about $100,000, according to Tirman, including about $45,000 for publicity and travel. That means that nearly half of the study's funding came from an outspoken billionaire who has repeatedly criticized the Iraq campaign and who spent $30 million trying to defeat Bush in 2004.

• Partisan considerations. Soros is not the only person associated with the Lancet studies who had one eye on the data and the other on the U.S. political calendar. In 2004, Roberts conceded that he opposed the Iraq invasion from the outset, and -- in a much more troubling admission -- said that he had e-mailed the first study to The Lancet on September 30, 2004, "under the condition that it come out before the election." Burnham admitted that he set the same condition for Lancet II. "We wanted to get the survey out before the election, if at all possible," he said.

"Les and Gil put themselves in position to be criticized on the basis of their views," Garfield concedes, before adding, "But you can have an opinion and still do good science." Perhaps, but the Lancet editor who agreed to rush their study into print, with an expedited peer-review process and without seeing the surveyors' original data, also makes no secret of his leftist politics.

As Say Anything's Ken McCracken observes, "Once again, in the tradition of Walter Duranty, we find the left is most eager to play games with numbers of mass casualties to suit their ideological biases and project their agenda." The Lancet studies buzz was -- like the NY Times Magazine's April 2006 cover piece falsely claiming El Salvador was jailing women for having abortions -- another story "too good to fact check." Will the MSM ever learn? Tom Blumer of BizzyBlog is skeptical:

None of these issues kept Old Media from jumping on the story three weeks before the 2006 mid-term elections. . .

This should be a lesson to Old Media that a little digging is in order when something so out of line with previous reports shows up. But it's one that probably won't be learned -- at least when outlier studies like Lancet's fit their advocacy template.

Agreed--but the good news is that such bias can be conservatives' secret weapon.

MORE:

A WHO study covering the same time period estimates one fourth the Lancet numbers, though the NY Times somehow spins the result as "higher". And Michael commenting at Confederate Yankee reminds all:

However many Iraqis have died in the present war, the vast majority of them have been killed by the enemy, not by our forces. It is the enemy who have waged a relentless campaign of barbaric murders of unarmed civilians.

(via NewsBusters, The Weekly Standard)