discussion-group — Blog — Penn Science Policy and Diplomacy Group

by Ian McLaughlin

On the 24^th of October, the Penn Science Policy Group met to discuss the implications of a new mechanism by which individuals can essentially take part in the peer review process. The group discussion focused on a particular platform, PubPeer.com, which emerged in 2012 and has since become a topic of interest and controversy among the scientific community. In essence, PubPeer is an online forum that focuses on enabling post-publication commentary, which ranges from small concerns by motivated article readers, to deeper dives into the legitimacy of figures, data, and statistics in the publication. Given the current state of the widely criticized peer-review process, we considered the advantages and disadvantages of democratizing the process with the added layer of anonymity applied to reviewers.

PubPeer has been involved in fostering investigations of several scandals in science. Some examples include a critical evaluation of papers published in Nature 2014 entitled Stimulus-triggered fate conversion of somatic cells into pluripotency [1]. The paper described a novel mechanism by which pluripotency might be induced by manipulating the pH environments of somatic cells. However, following publication, concerns regarding the scientific integrity of published experiments were raised, resulting in the retraction of both papers and an institutional investigation.

Subsequently, the publications of a prolific cancer researcher received attention on PubPeer, ultimately resulting in the rescission of a prestigious position at a new institution eleven days before the start date due, at least in part, to PubPeer commenters contacting faculty at the institution. When trying to return the professor’s former position, it was no longer available. The professor then sued PubPeer commenters, arguing that the site must identify the commenters that have prevented a continued career in science. PubPeer, advised by lawyers from the ACLU working pro-bono, is refusing to comply – and enjoy the support of both Google and Twitter, both of which have filed a court brief in defense of the website [2].

Arguably at its best, PubPeer ostensibly fulfills an unmet, or poorly-met, need in the science publication process. Our discussion group felt that the goal of PubPeer is one that the peer review process is meant to pursue, but occasionally falls short of accomplishing. While increased vigilance is welcome, and bad science – or intentionally misleading figures – should certainly not be published, perhaps the popularity and activity on PubPeer reveals a correctable problem in the review process rather than a fundamental flaw. While the discussion group didn’t focus specifically on problems with the current peer review process – a topic deserving its own discussion [3] – the group felt that there were opportunities to improve the process, and was ambivalent that a platform like PubPeer is sufficiently moderated, vetted, and transparent in the right ways to be an optimal means to this end.

Some ideas proposed by discussion participants were to make the peer-review process more transparent, with increased visibility applied to the reasons a manuscript is or is not published. Additionally, peer-review often relies upon the input of just a handful of volunteer experts, all of whom are frequently under time constraints that can jeopardize their abilities to thoroughly evaluate manuscripts – occasionally resulting in the assignment of peer review to members of related, though not optimally relevant, fields [4]. Some discussion participants highlighted that a democratized review process, similar to that of PubPeer, may indeed alleviate some of these problems with the requirement that commenters be moderated to ensure they have relevant expertise. Alternatively, some discussion participants argued, given the role of gate-keeper played by journals, often determining the career trajectories of aspiring scientists, the onus is on Journals’ editorial staffs to render peer review more effective. Finally, another concept discussed was to layer a 3rd party moderation mechanism on top of a platform like PubPeer, ensuring comments are objective, constructive, and unbiased.

The concept of a more open peer review is one that many scientists are beginning to seriously consider. In Nature News, Ewen Callaway reported that 60% of the authors in Nature Communications agreed to have publication reviews published [7]. However, while a majority of responders to a survey funded by the European Commission believed that open peer review ought to become more routine, not all strategies of open peer review received equivalent support.

[7]

Ultimately, the group unanimously felt that the popularity of PubPeer ought to be a signal to the scientific community that something is wrong with the publication process that requires our attention with potentially destructive ramifications [5]. Every time a significantly flawed article is published, damage is done to the perception of science and the scientific community, and at a time when the scientific community still enjoys broadly positive public perception [6], now is likely an opportune time to reconsider the peer-review process – and perhaps learn some lessons that an anonymous post-publication website like PubPeer might teach us.

References

1) PubPeer - Stimulus-triggered fate conversion of somatic cells into pluripotency. (n.d.). Retrieved November 25, 2016, from https://pubpeer.com/publications/8B755710BADFE6FB0A848A44B70F7D

2) Brief of Amici Curiae Google Inc. and Twitter Inc. in Support of PubPeer, LLC. (Michigan Court of Appeals). https://pubpeer.com/Google_Twitter_Brief.pdf

3) Balietti, S. (2016). Science Is Suffering Because of Peer Review’s Big Problems. Retrieved November 25, 2016, from https://newrepublic.com/article/135921/science-suffering-peer-reviews-big-problems

4)Arns M. Open access is tiring out peer reviewers. Nature. 2014 Nov 27;515(7528):467. doi: 10.1038/515467a. PubMed PMID: 25428463.

5) Jha, Alok. (2012). False positives: Fraud and misconduct are threatening scientific research. Retrieved November 25, 2016, from https://www.theguardian.com/science/2012/sep/13/scientific-research-fraud-bad-practice

6) Hayden, E. C. (2015, January 29). Survey finds US public still supports science. Retrieved November 25, 2016, from http://www.nature.com/news/survey-finds-us-public-still-supports-science-1.16818

7) Callaway E. Open peer review finds more takers. Nature. 2016 Nov 10;539(7629):343. doi: 10.1038/nature.2016.20969. PubMed PMID: 27853233

**Link for live streaming of this event can be found here**

by Hannah Shoenhard, Jamie DeNizio, and Michael Allegrezza

Craig Spencer, a New York City doctor, tested positive for Ebola on October 23. The story broke online the same day, and by the next morning, tabloids were plastered with images of masked and gowned health workers with headlines such as Bungle Fever and Ebola! Late-night comedy, Twitter, local news: the story was inescapable, the hysteria palpable. All in all, only eleven Ebola patients were treated on U.S. soil. But the media’s reaction affected the lives of anyone who watched television or had an internet connection.

The Ebola epidemic in Africa has died down. Liberia is Ebola-free, while Sierra Leone and Guinea continue to report cases in the low single digits per week. Most promisingly, a new vaccine has been shown to be highly effective in a clinical trial. Given the vaccine, it seems that the likelihood of future epidemics on the scale of the one in 2014 is low. But especially during the early days of the epidemic, miscommunication and mistrust of international public health workers slowed the medical response and exacerbated the epidemic. And, as the reaction to the New York City case shows us, this problem is not unique to West African countries.

Even if the threat from Ebola in particular is under control, infectious disease is endemic to civilization. Knowing that new epidemic threats can emerge at any time, important questions need to be considered.

How prepared are Philadelphia’s institutions to communicate with the public in the event of a future epidemic? What specific challenges were successfully or unsuccessfully addressed during the Ebola crisis that could provide learning points going forward? Are there successful models or case studies for handling communication during epidemics that are worth emulating?

These questions will be up for debate on Wednesday at the University of Pennsylvania in a forum open to the public. The event will be held in the Penn bookstore (3601 Walnut St.) upstairs meeting room from 5:30 to 7 p.m. on Wednesday, November 4.

The event is hosted by two graduate student groups at Penn, the Emerging Leaders in Science and Society (ELISS) Fellows and the Penn Science Policy Group with the goal of fostering collaborative ideas to develop effective channels to manage trust, fear, and accurate communication during potential future epidemics.

On the panel for the forum will be three innovators in communicating public health issues, Dr. Mitesh Patel, MD, MBA, MS, James Garrow, MPH, and Dr. Giang T. Nguyen, MD, MPH, MSCE. Moderating the discussion will be Dr. Max King, Ph.D., Associate Vice Provost for Health and Academic Services at Penn.

Community members are encouraged to attend the forum with questions and comments. People can also watch a live stream of the event (check our twitter page for the link) and submit questions via twitter with #EpidemicPhilly.

Biographies of the panelists and moderator

Mitesh Patel

Mitesh S. Patel, MD, MBA, MS is a board-certified general internist, physician scientist, and entrepreneur. He is an Assistant Professor of Medicine and Health Care Management at the Perelman School of Medicine and The Wharton School at the University of Pennsylvania. His work focuses on leveraging behavioral economics, connected health approaches and data science to improve population health and increase value within the health care delivery system.

As a physician-scientist, Mitesh studies how we can utilize innovative technology and connected health approaches to passively monitor activity and how we can use health incentives to motivate behavior change. His prior work has been published in NEJM, JAMA, the Annals of Internal Medicine and featured in the New York Times, NPR, and CNN. Mitesh also co-founded, Docphin, a startup that strives to improve the application of evidence-based medicine into clinical practice.

Mitesh holds undergraduate degrees in Economics and Biochemistry as well as a Medical Doctorate from the University of Michigan. He obtained an MBA in Health Care Management from The Wharton School and an MS in Health Policy Research from the Perelman School of Medicine at the University of Pennsylvania. Mitesh completed his internal medicine residency training and the Robert Wood Johnson Clinical Scholars Program at the University of Pennsylvania.

James Garrow

James Garrow, MPH is a nationally recognized proponent and advocate for the use of social media and digital tools in the execution of public health activities. His role as the Director of Digital Public Health in Philadelphia is among the first in the country charged with using new digital tools and techniques like social media, crowdsourcing, and big data utilization. He also provides media relations support for the Philadelphia Department of Public Health.

An accomplished public speaker and noted thought leader, Jim has been invited to and spoken at conferences across the US on social media use in public health and emergency response. He is an active social media user, maintaining two regularly scheduled Twitterchats and a blog on crisis and emergency risk communications.

Jim obtained a B.S. in Applied Sociology from Drexel in 2001 and a Master’s of Public Health from Temple in 2011.

Giang T. Nguyen

Dr. Nguyen, MD, MPH, MSCE, is an Assistant Professor in the School of Medicine, Department of Family Medicine and Community Health at Penn. He is also Chair of the MD-MPH Advisory Committee and a member of the MPH Program Curriculum Committee.

Dr. Nguyen leads the Penn Asian Health Initiatives. His research focus is in Asian immigrant health with concentrations in cancer control, disease prevention, and community-based participatory research. His community engagement work has included outreach to Vietnamese and other Southeast Asian refugees, health fairs and immunization clinics, cancer education workshops, advocacy, HIV/AIDS, and LGBT issues. He serves on boards and advisory committees for several Asian serving organizations, including the Asian and Pacific Islander National Cancer Survivors Network.

Dr. Nguyen is also the Medical Director of Penn Family Care, the clinical practice of the University of Pennsylvania's Department of Family Medicine and Community Health. He provides direct care to adult and pediatric patients in the primary care setting and teaches medical students and family medicine residents. He also is a Senior Fellow of the Penn Center for Public Health Initiatives, where he is a core faculty member for the Penn MPH program.

Max King

Previously the coordinator of Penn State's University Scholars Program, Max King, Ph.D., MS, is now Associate Vice Provost for Health and Academic Services at The University of Pennsylvania.

Dr. King holds three degrees from Penn State: a B.S. in Biological Health, M.S. in Health Education, and Ph.D. from the Interdisciplinary Program in Educational Theory and Policy. His research focus is the multidimensional Methodology of Q-Analysis, or Polyhedral Dynamics, a higher-level structural analysis approach derived from algebraic topology.

Dr. King also held an appointment as an Affiliate Assistant Professor and a member of the graduate faculty in the Department of Administration, Policy, Foundations, and Comparative/International Education at Penn State. He taught educational foundations, comparative education, British education, research methods, and international education. He also has extensive experience in computer systems, developing mainframe and microcomputer research and thesis applications.