Another interesting point I noticed in the National Journal
article about the Lancet study (http://nationaljournal.com/njcover.htm)
is where it confirms that the study did not collect demographic
Lack of supporting data. The survey teams failed to collect
the fraud-preventing demographic data that pollsters routinely
gather. For example, D3 Systems, a polling firm based in Vienna,
Va., that has begun working in Iraq, tries to prevent chicanery
among its 100-plus Iraqi surveyors by requiring them to ask
respondents for such basic demographic data as ages and birthdates.
This anti-fraud measure works because particular numbers tend
to appear more often in surveys based on fake interviews and
data -- or "curb-stoning -- than they would in truly random
surveys, said Matthew Warshaw, the Iraq director for D3. Curb-stoning
surveyors might report the ages of many people to be 30 or 40,
for example, rather than 32 or 38. This type of fabrication
is called "data-heaping," Warshaw said, because once
the data are transferred to spreadsheets, managers can easily
see the heaps of faked numbers.
I remembered the issue of demographic data came up almost right
after the study was released in 2006 in this WSJ article by a
Though in this case I think Moore's point was not its utility
as a defense against fraud by surveyors, but rather its utility
as a test of representativeness of the sample drawn.
In any case, Les Roberts dismissed this criticism in haste by
accusing Moore of "fabrication":
I am not surprised at his rejection of my suggestion that
the references in a web report explaining the methodology for
lay people and reporters was not the same as the references
in our painstakingly written peer reviewed article. What is
striking is Mr. Moore's statement that we did not collect any
demographic data, and his implication that this makes the report
This is curious because, not only did I tell him that we
asked about the age and gender of the living residents in the
houses we visited, but Mr. Moore and I discussed, verbally and
by e-mail, his need to contact the first author of the paper,
Gilbert Burnham, in order to acquire this information as I did
not have the raw data. I would assume that this was simply a
case of multiple misunderstandings except our first report in
the Lancet in 2004 referenced in our article as describing the
methods states, ".interviewees were asked for the age and
sex or every current household member."
Thus, it appears Mr. Moore had not read the description of
the methods in our reports. It is not important whether this
fabrication that "no demographic data was collected"
is the result of subconscious need to reject the results or
whether it was intentional deception. What is important, is
that Mr. Moore and many others are profoundly uncomfortable
that our government might have inadvertently triggered 650,000
Through this handy deflection of a valid criticism, Moore was
of course (as intended by Roberts) dismissed by Roberts' supports
as an idiot, a liar and even worse: a Republican. So Moore's criticisms
went down the memory hole. Yet another criticism evaded and swept
aside, even while it was actually Roberts who was lying (successfully).
Roberts' false dichotomy/ad hominem is a nice kicker at the end
too. But then maybe Moore was just uncomfortable with sloppy and
biased "science" being promoted by people who give only
slippery (or intentionally deceitful, as in this case) answers
to critical questions?
I'd almost forgotten about this, but it's nice that the NJ article
provides some closure to it. Though it also makes me wonder how
many other criticisms have been evaded by Roberts and co just
lying them away. There could be a long laundry list of these.
One good candidate seems to be how, only after the Main Street
Bias critique emerged, suddenly the study did not use the selection
methodology that was published. According to Burnham in NJ, and
Roberts previously, the one that was in their "painstakingly
written peer reviewed article" was not what was used. There
was apparently an "editing error" painstakingly missed,
which resulted in the wrong methodology getting reviewed and published
in Lancet. Some other (as yet unknown) method was supposedly used
that would have conveniently eliminated the prospect of Main Street
Bias, or so became the story after the MSB criticism emerged.
I wonder if these claims are as reliable as those from Roberts
about the demographic data.