PEERING AT DOUBLE-BLIND PEER REVIEWby James Peterson
Any search engine on the internet can instantly amass a heap of opinions and claims on almost any subject. Perspectives on Science and Christian Faith (PSCF), as a peer-reviewed journal (and another publication of the American Scientific Affiliation), aims to do something much more than that. PSCF publishes essays on a wide variety of topics relevant to science and Christianity that are clear, accurate, well-informed, and bring something new to the conversation. The process PSCF uses to achieve that is called double-blind peer review.
When someone submits an essay, the editor-in-chief first reads it carefully to see if it meets the above standards closely enough to consider developing it for publication. If the essay has that potential, the editor identifies scholars who have special expertise on each part of the proposed essay. There are typically three reviewers, but if the essay is highly interdisciplinary, it may require as many as five or six. The editor then contacts each prospective reviewer and asks if he/she would be willing to review an essay on the topic at hand, and usually gives the potential reviewer some sense of why the editor recognizes that they have the needed expertise and thinks that they would find the essay interesting. The editor then prepares a copy of the essay that does not reveal the name or identity of the author (“blinds” the reviewer). This way, the reviewer will not be influenced by the author’s stellar reputation or lack thereof; the essay will be evaluated purely on the basis of what it achieves.
At PSCF we ask reviewers to send their evaluations back to the editor within three or four weeks. We expect the reviewer to assess how well the essay expresses its argument with clarity, accuracy, and documented awareness of the applicable conversation, and whether it is indeed making a new and important contribution. The bar is high. Essays never meet all standards completely on the first submission. After reviewers have assessed the essay, their feedback and specific suggestions on how it can be improved are returned to the editor. The editor then takes those suggestions from each of the peer reviewers and edits them into a manageable set of advice for the author. At this point in the process, the editor communicates to the author that the essay is not going to be publishable, or that revisions and further development of the piece are required before it is publishable. The names of the reviewers are not revealed to the author (the author is also “blind” in that sense), allowing the reviewers to speak directly and frankly and use their best judgment. They are motivated to do incisive and fair critique because the system judges their essays too, and the editor will see their peer review alongside that of others. Reputation is hard to earn and easy to lose in academia. The system is called double-blind because, as we saw, the reviewers do not know who the author of the article is, and the author does not know who is writing the reviews. This encourages the argument to be evaluated in both directions solely on the basis of its merit. Reviewers will not necessarily agree with the author’s conclusions, but they can articulate what would be needed for the most effective contribution. Naturally, sometimes they will disagree with each other as well, although it is quite striking how often there is a clear consensus in their critiques. Granted, no one reviewer catches everything. Individual reviewers will often spot concerns that other reviewers did not. The combination of multiple peer reviewers giving their best advice is a tremendous help to the editor, as well as to the author who is willing, indeed excited, about improving her work. As the author rewrites the essay taking into account the suggestions of the reviewers, the editor decides when indeed the article has satisfied all the comments in the critique, and standards to be published. This triggers the type of letter I most enjoy writing: that the essay is clear, accurate, well-informed, and making an important contribution – and so will appear in the journal. This process takes more time than putting up a blog post on the web as soon as one thinks of it, but if the editor keeps everything moving, it’s usually not that much of a delay. The pay-off for all the extra effort is the quality of what is then available to the reader. The heroes in the process are the peer reviewers, who are not paid directly, nor do they immediately see their names in print. They can report at their home institution that they did peer review for a particular journal. That carries appreciation and prestige, since they were singled out as experts. Also, PSCF does publish a list of peer reviewers to thank them, far enough after doing peer review so that they cannot be tied for sure to any particular review. The reviewer probably also trusts the editor that the essay will offer something new and interesting, and they know that their publications have benefited from peer review as well. There is a duty of reciprocity to aid a process that has been so helpful to them. Most importantly, peer review is a real service to the readers and the Kingdom. It is an opportunity to encourage best effort toward better understanding. A well working peer review system produces dramatically superior articles for everyone involved, and that is, after all, the objective. 
|
James C. Peterson, PhD (University of Virginia), is the Schumann Professor and Director at the Benne Center for Religion & Society of
Roanoke College. He is also the ethicist for both the Lewis Gale Medical Center and the Virginia Tech Carilion School of Medicine. Specifically concerning God and nature, he is an ordained minister, a recipient of a genetics research fellowship funded by NIH, a Fellow of the ASA and ISSR, former president of the CSCA, author of books such as Changing Human Nature (Eerdmans), and Editor-in-Chief of the academic journal Perspectives on Science and Christian Faith |