I've been following with some interest, a debate on introducing double blind reviews (DBR) [1] in CS conferences/journals, happening on some CS blogs. They are mostly talking about theoretical CS, but it's valid generally. It started here (
http://kdphd.blogspot.com/2009/02/double-blind-reviewing.html), wherein the author of the post provides some very good rebuttals to general objections to DBR. The context is the perceived bias specially against female authors, grad students, young and unknown researchers. Then there were posts in support of DBR (with some caveats) by Michael Mitzenmacher (
http://mybiasedcoin.blogspot.com/) and Suresh Venkatasubramanian (
http://geomblog.blogspot.com/2009/03/double-blind-review-again.html). There are many interesting comments to these posts as well.
While there are many who support DBR, there are many more who oppose it acutely. Some reasons for the opposition are valid, the important ones being: (1) one has to freeze research articles until published; this deters dissemination of knowledge (2) it makes declaring conflicts of interests in programme committees tricky (3) it may increase the number of submissions, as many people can submit their marginal work without loss of reputation at stake. However, there have been some good suggestions to address these problems from the DBR supporters.
What is amazing is the way this topic seems to have touched a few nerves among people who want the status quo. The arguments against DBR, range from blatant denial to presumption to totally fallacious. There are claims as the following - "Double blind reviewing is the hallmark of shallow "engineering" type disciplines, where pursuit of fame and power, rather the advancement of science, is the driving force." Whereas some totally deny that anything is wrong with the system. And actually say that theory people, unlike others, are too smart to have any biases whatsoever. None of them is joking!
What is even more revealing is that, some admit there are biases, specially in favour of well known names, and against not well known names; but they argue, subtly or eloquently, that it is only fair that this happens. When an involved proof is submitted, if it comes from a famous researcher, that is enough to establish the credibility, whereas if the name is unknown, the reviewer should be skeptical -- this is also an argument given. There is some very odd reasoning posted under the shroud that theorists are artistes, and artistes work should be taken in context, and brand matters etc. (
http://weblog.fortnow.com/2009/03/you-can-separate-art-from-artist.html). Some of these remind of high-school kids discussing why a particular teacher is partial towards a particular student, and how it is only logical that the bias exists because the student has established herself as an outstanding one. While there is some validity in associating credibility with authority, if a program committee cannot decide on the quality of a work without knowing who it comes from, there is a problem with the review process. May be overloading or lack of incentive, or something.
One thing that is conspicuous from its absence is the 'it doesn't matter to me if it's a DBR, let's give it a try' view point from the non-supporters. Given this and the impassioned tirade against DBR, it is evident that there is a huge incentive for many to stick with status quo. For me, that itself is sufficient reason to support DBR. Indeed, some comments say things like 'I am probably okay with the bias as it works for me'. Some others say, 'Anyway, it's such a small community. We all know who is working on what and what is its worth.' It's odd that they don't realize how strongly this suggests entrenchment!
All in all, it is the fervent opposition to DBR, more than the excellent points in its support, that has convinced me that the conference reviewing systems are more broken than I had imagined. I hope that the problems associated with DBR will be addressed and that soon everyone will start using it.
[1] Double blind reviewing is the process of reviewing papers in which both the authors and the reviewers of a paper remain anonymous to each other. This is the alternative to single blind reviewing, in which the reviewers know the names of the authors, but authors don't know who the reviewers are.
Print