Peer Review and the Public’s Health

Essentially all clinical practice, public health administration, and health policy should be based on research published in peer-reviewed biomedical, health science, and policy journals. The textbooks our students read and our colleagues consult also are based on what has been published in those journals. Thus, it is critical that all peer-reviewed published articles are based on valid data, good science, and sound reasoning.

When reflecting on my more than 17 years as an editor, including 11 and a half years as the editor-in-chief of JAMA, the Journal of the American Medical Association, I agree with my colleague-editor Drummond Rennie that

there seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.¹

This statement was made in 1986 in the announcement of the first International Congress on Peer Review in Biomedical Publication that was initiated and led by Rennie.

Sadly, there are relatively few barriers to publishing anything. But when one considers that Medline, the US National Library of Medicine’s premier bibliographic database, contains approximately 21 million records from 5,600 journals worldwide, published in about 40 languages, with 700,000 citations added in 2013 (http://www.nlm.nih.gov/pubs/factsheets/medline.html), it becomes clear that it would be virtually impossible to ensure the validity of every publication. Validity can be assured only through the integrity and industry of investigators/authors who would not publish false or inferior information for any reason. The process that biomedical, health science, and policy journals (including The Milbank Quarterly) have used to ensure the best possible publication outcomes is peer review, which in this case is the evaluation of an article by one or more persons whose expertise is similar to that of the investigator(s)/author(s).

The history of peer review’s evolution is not clear, although it probably began when individuals started discussing what was perceived to be new knowledge.Whenever and wherever it was initiated, the system of peer review that exists today probably began in the mid-20th century.²

Currently, the vast majority of articles published in biomedical, health science, and policy journals have undergone some form of peer review. This ranges from the editor acting as the sole reviewer, making all decisions about what is published, to the editor seeking the advice of many persons who have specific expertise regarding the manuscript in question.

It is important to note that the highest-ranking editor (editor-in-chief) makes all final decisions regarding what is published in her or his journal. Other deputy, associate, or assistant editors (internal reviewers) or reviewers from outside the journal are advisers. Therefore it is incumbent on the editor-in-chief to gather the best possible advice regarding what should be published. Not only does her or his reputation and that of the journal rest on the quality of what is published. More important, the publication might contain information that could be used to improve or harm the health of, literally, millions of people. I always felt that my role as a physician was no different if I was directly caring for a patient or if what I wrote, edited, or published might be used by other clinicians to care for patients.

So if peer review cannot ensure the validity of what is published, let alone provide assurance that patients will not be harmed by the information in the publication, what value is there in the peer-review process? Essentially, peer review is a form of self-regulation in an attempt to maintain standards of quality, competence, and significance; to improve performance of the investigation and authorship; and to provide credibility.

The internal peer review by the other editors of a journal provides expertise regarding the general scientific quality of the manuscript, including its methodology, data, data analysis, discussion, and conclusions. In many cases, these internal reviewers also have some expertise in the specific area covered in the manuscript.

The external peer review by persons with specific expertise in the medical area discussed in the manuscript is essential because it is impossible for one person to know everything about all subjects. Having several so-called topic peer reviewers thus is valuable even if they disagree, and if one peer reviewer has a bias, it will be diluted by the views of the other reviewers. Good reviewers are invaluable, and some even argue that they should receive some monetary reimbursement in addition to learning from the manuscript and from other reviewers, the Continuing Medical Education Credits awarded by some journals, and the credit given to reviewers by some institutions. This, however, is a thorny negotiation, best left for another day.

After publication, the peer review continues by the readers of the article. These individuals can write letters to the editor or discuss their views privately with colleagues and students or publicly at meetings or conferences. In fact, this post-publication peer review has led to the correction or even the retraction of articles, from even the most prestigious journals.

Some individuals argue that their studies, especially those reporting negative results, are less likely to be published by journals. I am not aware of any good studies that have shown this to be true, and I know of at least one that shows it to be untrue.³ While researchers are more likely to submit papers showing positive results, it is not clear that editors are more likely to publish them.

Conversely, however, there is excellent evidence that for-profit companies have been reluctant to publish or even register negative clinical trials, stimulating the International Committee of Medical Journal Editors to force the registration of these trials as soon as the first subject is enrolled.⁴

Unfortunately, some institutions have begun to bypass peer review by publishing their papers on their own websites. This form of “gray literature” only adds to the current era of disruption, lack of editing, and aggregation of news sources. The authors of such papers certainly do not always provide extensive evidence of the validity of their conclusions.

Clearly, the peer-review process needs to be constantly evaluated, studied, and improved, which is what has been done formally at least since 1986 when Rennie, along with several other journal editors, announced the first International Congress on Peer Review in Biomedical Publication. This initiative sought to encourage editors, reviewers, authors, and others to examine the process of peer review as they might examine science itself; facilitate evaluation and discussion; and foster the publication of such research results. There have now been 7 congresses from 1989 to 2013, with another planned for 2017.

Peer review is not perfect, but if I may paraphrase the sage Winston Churchill, it has been said that peer review is the worst form of ensuring the accuracy of published articles except for all the others that have been tried.

References

1. Rennie D. Guarding the guardians. JAMA. 1986;256(17):2391- 2392.
2. Burnham JC. The evolution of editorial peer review. JAMA. 1990;263(10):1323-1329.
3. Olson CM, Rennie D, Cook D, et al. Publication bias in editorial decision making. JAMA. 2002;287(21):2825-2828.
4. DeAngelis CD, Drazen JM, Frizelle FA, et al. Clinical trial registration. JAMA. 2004;292(11):1363-1364.

Author(s): Catherine D. DeAngelis

Read on Wiley Online Library

Volume 93, Issue 2 (pages 247–250)
DOI: 10.1111/1468-0009.12118
Published in 2015

About the Author