Anonymous peer review has the power to transform. It can turn trust into suspicion; it can be uplifting or leave you without a leg to stand on. It can rain on your argumentative parade or let your words shine.
Anonymity ideally should ensure fairness. Very often, however, the opposite is the case: “This dishonest procedure brings out the worst in people,” argues Geert in a blog entry called ‘Let’s end anonymous peer review’. Not being identifiable, some scholars take advantage of their anonymity and lash out against someone who might be too close to or too far from their own topic of research, too empirical or too theoretical for their own way of reasoning, too similar or too different from their own scholarly self-conception. Anonymous peer review, argue the editors of The New Atlantis in their post ‘Rethinking Peer Review’, “fails to achieve its core objective: quality control.”
The problem with anonymous peer review is that not only do the reviewers remain anonymous, but very often their criteria for approving, criticizing or rejecting another persons’ work remain hidden, too.
The Peer Review Survey found that scholars felt a lack of guidance when it comes to writing a comprehensive review.
Having been asked recently to write a review for the new Asia Pacific World, I was pleasantly surprised to see that the guidance sheet forwarded by the editors, alongside the article I was going to review, focused at first only on selection criteria concerning the article (asking concrete questions like “Does the article present its results/discussion in a clear and concise way, supporting the conclusion?”). Most of the sheet was devoted to helping the reviewer write a good review. The guiding questions were grouped into three categories, namely validation, evaluation, and style.
With the journal editors’ permission, I am citing these questions here in full because I think they are really helpful for everyone writing reviews.
Validation
- Does your review cite appropriate evidence to support your score and reviewer recommendations (to accept with minor revisions/ to return with major revisions/ to reject outright)?
- Is your review thorough and comprehensive?
- Is your review objective and balanced?
Evaluation
- Will your review assist the authors in improving their manuscript?
- Will your review assist the editors to make a decision?
- For articles that require revision, have you given clear recommendations with specific suggestions for improvement if required?
- For articles that will be rejected outright, have you given clear reasons why the article is being rejected?
Style
- Have you ensured a courteous yet constructively critical tone in your review?
- Have you ensured that you do not identify yourself by your comments, or by the name of your report when submitting as a Word file?
While some have argued that peer review is exploitation of our voluntary labor by for-profit publishers, others have said that “[r]eviewing is a good way to keep up with literature and sharpen your own writing.” Rex from Savage Minds wrote a while back that we should see peer review as a chance to cultivate virtues: mindfulness, honesty, tact, precision, and respect. Kerim added that one of the reasons why we agree to review other people’s work is because we see it “as a form of service to our community”. It would probably be difficult for a journal to quickly instill such virtues or a willingness to provide community service in their reviewers; but these questions, I think, are not only practical guidance, but can also help us reflect on the responsibilities we share as scholars.
And for those who feel less karmic about peer review, there is always #shitmyreviewerssay @yourpapersucks.
Very good recommendations – though normally you’d think that’s just common sense and a minimum level of civilised courtesy. My rule of thumb is that even if you don’t agree with the argument, you should always write it in a tone that you would also employ face-to-face (even if some drafts are sometimes truly exasperating).