The Fund supports networks of state health policy decision makers to help identify, inspire, and inform policy leaders.
The Milbank Memorial Fund supports two state leadership programs for legislative and executive branch state government officials committed to improving population health.
The Fund identifies and shares policy ideas and analysis to advance state health leadership, strong primary care, and sustainable health care costs.
Keep up with news and updates from the Milbank Memorial Fund. And read the latest blogs from our thought leaders, including Fund President Christopher F. Koller.
The Fund publishes The Milbank Quarterly, as well as reports, issues briefs, and case studies on topics important to health policy leaders.
The Milbank Memorial Fund is is a foundation that works to improve population health and health equity.
June 2015 (Volume 93)
Quarterly Article
Catherine D. DeAngelis
December 2024
Dec 19, 2024
Back to The Milbank Quarterly
Essentially all clinical practice, public health administration, and health policy should be based on research published in peer-reviewed biomedical, health science, and policy journals. The textbooks our students read and our colleagues consult also are based on what has been published in those journals. Thus, it is critical that all peer-reviewed published articles are based on valid data, good science, and sound reasoning.
When reflecting on my more than 17 years as an editor, including 11 and a half years as the editor-in-chief of JAMA, the Journal of the American Medical Association, I agree with my colleague-editor Drummond Rennie that
there seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.1
This statement was made in 1986 in the announcement of the first International Congress on Peer Review in Biomedical Publication that was initiated and led by Rennie.
Sadly, there are relatively few barriers to publishing anything. But when one considers that Medline, the US National Library of Medicine’s premier bibliographic database, contains approximately 21 million records from 5,600 journals worldwide, published in about 40 languages, with 700,000 citations added in 2013 (http://www.nlm.nih.gov/pubs/factsheets/medline.html), it becomes clear that it would be virtually impossible to ensure the validity of every publication. Validity can be assured only through the integrity and industry of investigators/authors who would not publish false or inferior information for any reason. The process that biomedical, health science, and policy journals (including The Milbank Quarterly) have used to ensure the best possible publication outcomes is peer review, which in this case is the evaluation of an article by one or more persons whose expertise is similar to that of the investigator(s)/author(s).
The history of peer review’s evolution is not clear, although it probably began when individuals started discussing what was perceived to be new knowledge.Whenever and wherever it was initiated, the system of peer review that exists today probably began in the mid-20th century.2
Currently, the vast majority of articles published in biomedical, health science, and policy journals have undergone some form of peer review. This ranges from the editor acting as the sole reviewer, making all decisions about what is published, to the editor seeking the advice of many persons who have specific expertise regarding the manuscript in question.
It is important to note that the highest-ranking editor (editor-in-chief) makes all final decisions regarding what is published in her or his journal. Other deputy, associate, or assistant editors (internal reviewers) or reviewers from outside the journal are advisers. Therefore it is incumbent on the editor-in-chief to gather the best possible advice regarding what should be published. Not only does her or his reputation and that of the journal rest on the quality of what is published. More important, the publication might contain information that could be used to improve or harm the health of, literally, millions of people. I always felt that my role as a physician was no different if I was directly caring for a patient or if what I wrote, edited, or published might be used by other clinicians to care for patients.
So if peer review cannot ensure the validity of what is published, let alone provide assurance that patients will not be harmed by the information in the publication, what value is there in the peer-review process? Essentially, peer review is a form of self-regulation in an attempt to maintain standards of quality, competence, and significance; to improve performance of the investigation and authorship; and to provide credibility.
The internal peer review by the other editors of a journal provides expertise regarding the general scientific quality of the manuscript, including its methodology, data, data analysis, discussion, and conclusions. In many cases, these internal reviewers also have some expertise in the specific area covered in the manuscript.
The external peer review by persons with specific expertise in the medical area discussed in the manuscript is essential because it is impossible for one person to know everything about all subjects. Having several so-called topic peer reviewers thus is valuable even if they disagree, and if one peer reviewer has a bias, it will be diluted by the views of the other reviewers. Good reviewers are invaluable, and some even argue that they should receive some monetary reimbursement in addition to learning from the manuscript and from other reviewers, the Continuing Medical Education Credits awarded by some journals, and the credit given to reviewers by some institutions. This, however, is a thorny negotiation, best left for another day.
After publication, the peer review continues by the readers of the article. These individuals can write letters to the editor or discuss their views privately with colleagues and students or publicly at meetings or conferences. In fact, this post-publication peer review has led to the correction or even the retraction of articles, from even the most prestigious journals.
Some individuals argue that their studies, especially those reporting negative results, are less likely to be published by journals. I am not aware of any good studies that have shown this to be true, and I know of at least one that shows it to be untrue.3 While researchers are more likely to submit papers showing positive results, it is not clear that editors are more likely to publish them.
Conversely, however, there is excellent evidence that for-profit companies have been reluctant to publish or even register negative clinical trials, stimulating the International Committee of Medical Journal Editors to force the registration of these trials as soon as the first subject is enrolled.4
Unfortunately, some institutions have begun to bypass peer review by publishing their papers on their own websites. This form of “gray literature” only adds to the current era of disruption, lack of editing, and aggregation of news sources. The authors of such papers certainly do not always provide extensive evidence of the validity of their conclusions.
Clearly, the peer-review process needs to be constantly evaluated, studied, and improved, which is what has been done formally at least since 1986 when Rennie, along with several other journal editors, announced the first International Congress on Peer Review in Biomedical Publication. This initiative sought to encourage editors, reviewers, authors, and others to examine the process of peer review as they might examine science itself; facilitate evaluation and discussion; and foster the publication of such research results. There have now been 7 congresses from 1989 to 2013, with another planned for 2017.
Peer review is not perfect, but if I may paraphrase the sage Winston Churchill, it has been said that peer review is the worst form of ensuring the accuracy of published articles except for all the others that have been tried.
References
Author(s): Catherine D. DeAngelis
Read on Wiley Online Library
Volume 93, Issue 2 (pages 247–250) DOI: 10.1111/1468-0009.12118 Published in 2015
Catherine D. DeAngelis is Johns Hopkins University Distinguished Service Professor Emerita and professor emerita at the Johns Hopkins University Schools of Medicine (Pediatrics) and Public Health (Health Policy and Management), and editor-in-chief emerita of JAMA, where she served as the first woman editor-in-chief from 2000 to 2011. She received her MD from the University of Pittsburgh’s School of Medicine, her MPH from the Harvard Graduate School of Public Health, and her pediatric specialty training at the Johns Hopkins Hospital. She has authored or edited 12 books on pediatrics, medical education, and patient care and professionalism and has published over 250 peer-reviewed articles, chapters, and editorials. Her recent publications have focused on professionalism and integrity in medicine, conflict of interest in medicine, women in medicine, and medical education. DeAngelis is a member of the Institute of Medicine and a fellow of the American Association for the Advancement of Science and the Royal College of Physicians (United Kingdom). She currently serves on the advisory board of the US Government Accountability Office, is a member of the board of Physicians for Human Rights, and serves on the board of trustees of the University of Pittsburgh.
