Back to Messageboard | Home | Site Map  

Lancet demographic data "fabrication"  
Posted by sonny on January 7 2008, 20:17 » Uploaded 07/01/08 22:25  

Another interesting point I noticed in the National Journal article about the Lancet study ( is where it confirms that the study did not collect demographic data:

Lack of supporting data. The survey teams failed to collect the fraud-preventing demographic data that pollsters routinely gather. For example, D3 Systems, a polling firm based in Vienna, Va., that has begun working in Iraq, tries to prevent chicanery among its 100-plus Iraqi surveyors by requiring them to ask respondents for such basic demographic data as ages and birthdates. This anti-fraud measure works because particular numbers tend to appear more often in surveys based on fake interviews and data -- or "curb-stoning -- than they would in truly random surveys, said Matthew Warshaw, the Iraq director for D3. Curb-stoning surveyors might report the ages of many people to be 30 or 40, for example, rather than 32 or 38. This type of fabrication is called "data-heaping," Warshaw said, because once the data are transferred to spreadsheets, managers can easily see the heaps of faked numbers.

I remembered the issue of demographic data came up almost right after the study was released in 2006 in this WSJ article by a Steven Moore:

Though in this case I think Moore's point was not its utility as a defense against fraud by surveyors, but rather its utility as a test of representativeness of the sample drawn.

In any case, Les Roberts dismissed this criticism in haste by accusing Moore of "fabrication":

I am not surprised at his rejection of my suggestion that the references in a web report explaining the methodology for lay people and reporters was not the same as the references in our painstakingly written peer reviewed article. What is striking is Mr. Moore's statement that we did not collect any demographic data, and his implication that this makes the report suspect.

This is curious because, not only did I tell him that we asked about the age and gender of the living residents in the houses we visited, but Mr. Moore and I discussed, verbally and by e-mail, his need to contact the first author of the paper, Gilbert Burnham, in order to acquire this information as I did not have the raw data. I would assume that this was simply a case of multiple misunderstandings except our first report in the Lancet in 2004 referenced in our article as describing the methods states, ".interviewees were asked for the age and sex or every current household member."

Thus, it appears Mr. Moore had not read the description of the methods in our reports. It is not important whether this fabrication that "no demographic data was collected" is the result of subconscious need to reject the results or whether it was intentional deception. What is important, is that Mr. Moore and many others are profoundly uncomfortable that our government might have inadvertently triggered 650,000 deaths.[...]

Through this handy deflection of a valid criticism, Moore was of course (as intended by Roberts) dismissed by Roberts' supports as an idiot, a liar and even worse: a Republican. So Moore's criticisms went down the memory hole. Yet another criticism evaded and swept aside, even while it was actually Roberts who was lying (successfully).

Roberts' false dichotomy/ad hominem is a nice kicker at the end too. But then maybe Moore was just uncomfortable with sloppy and biased "science" being promoted by people who give only slippery (or intentionally deceitful, as in this case) answers to critical questions?

I'd almost forgotten about this, but it's nice that the NJ article provides some closure to it. Though it also makes me wonder how many other criticisms have been evaded by Roberts and co just lying them away. There could be a long laundry list of these.

One good candidate seems to be how, only after the Main Street Bias critique emerged, suddenly the study did not use the selection methodology that was published. According to Burnham in NJ, and Roberts previously, the one that was in their "painstakingly written peer reviewed article" was not what was used. There was apparently an "editing error" painstakingly missed, which resulted in the wrong methodology getting reviewed and published in Lancet. Some other (as yet unknown) method was supposedly used that would have conveniently eliminated the prospect of Main Street Bias, or so became the story after the MSB criticism emerged.

I wonder if these claims are as reliable as those from Roberts about the demographic data.

COMMENTS Post comment


Comment 01 – Zepezauer January 08 2007, 08:47

In interviews, Les Roberts is fond of describing himself as an "advocate". Credible scientists would put science before advocacy, but Roberts appears to have had something akin to a religious conversion, hence the "advocate" label.

The truth about war is the case against war. The truth is all you need. It's bad enough in itself. You don't need to spin it into something quantitatively worse - always a counterproductive move, as spin has a way of being exposed in the long run.

Comment 02 – MinaM January 08 2007, 10:13

There are some scientific concerns that I've read about the Lancet in various articles, but I'm going to give it the benefit of the doubt unless these coalesce into a strong scientific consensus that something is seriously wrong with it. Meanwhile it's an educational experience reading all the posts on this issue. So much better than watching Newsnight. :)

Comment 03 – Several Famous Epidemiologists January 08 2007, 17:57

Les Roberts' comment (reported by Moore in the linked article) "that the appendices were written by a student and should be ignored" is curious. The appendices Moore was talking about (in the longer account of the JHU study than that published in the Lancet) take up pages 13-25 in a 25-page report, and contain much of importance, including details of methodology, data collection, etc. This was all written by students and is to be ignored?



Post comment

Display email
  Don't display    Display
Lifespan of comment   Delete after 3 weeks    Keep permanent if post is permanent  


Optional link URL:
eg ""
Optional link text:
eg "Media Hell"


  Messageboard Back to top