Friday, September 29, 2006

On Peer Review

When a distinguished professor like Philip Johnson or Michael Behe makes comments contrary to Darwinian theory, one can count on the response "what have they ever published in a peer reviewed biological journal?" Unfortunately, most do NOT understand that this is simply the logical fallacy called "appeal to authority," and thus I--having been on both sides of the peer review process--feel somewhat compelled to explain exactly why "peer review" has more or less the value of a Reichsmark in Weimar Germany.

In a nutshell, peer review guarantees nothing because it's an inherently political process, and this is especially so in politically charged areas like biology. What is unpopular is often excluded from peer reviewed journals. I've seen it happen. On the other hand, garbage research is often included in peer reviewed journals simply due to professional connections. I've seen that happen, too.

A classic case of the failure of peer review is shown in the recent government "study" of research linking abortion with breast cancer. 28 peer reviewed studies were reviewed by 100 workers in the field, and 16 of them were thrown out based on a perceived methodological error. What we clearly have here is two sets of peer review directly contradicting each other--peers selected by the same group of researchers, no less.

Whatever side of the debate one falls upon, one must concede that peer review is decidedly fallible, and all the more so as the issue becomes politically charged. Insisting on "peer review" instead of presenting evidence is a sad substitute for argument.

2 comments:

Matt said...

That's interesting. A few months ago my friend showed me an article that was accepted and published by a supposedly peer reviewed journal. The article used lots of fancy words and scientific jargon but what it actually said was that the laws of physics were mere social conventions that could be changed by society if desired.

The article was written by a guy trying to make a point and I think he succeeded.

Bike Bubba said...

That's an even better example than I was able to provide. Now that you mention it, I do remember that, and all I can say is "well done!"

Not that there is no value to peer review when people understand and appreciate their task. However, when it becomes politicized, watch out.