A multi-disciplinary perspective on emergent and future innovations in peer review

Tennant JP, Dugan JM, Graziotin D, Jacques DC, Waldner F, Mietchen D, Elkhatib Y, Collister LB, Pikas CK, Crick T, Masuzzo P, Caravaggi A, Berg DR, Niemeyer KE, Ross-Hellauer T, Mannheimer S, Rigling L, Katz DS, Greshake B, Pacheco-Mendoza J, Fatima N, Poblet M, Isaakidis M, Erwin Irawan D, Renaut S, Madan CR, Matthias L, Nørgaard Kjær J, O'Donnell DP, Neylon C, Kearns S, Selvaraju M, Colomb K. F1000 Research 6 :1151 (2017).

Abstract

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

BibTeX

 @article{Tennant:2017pr,
  Author = {Jonathan P Tennant and Jonathan M Dugan and Daniel Graziotin and Damien C Jacques and Fran\c{c}ois Waldner and Daniel Mietchen and Yehia Elkhatib and Lauren B.~Collister and Christina K Pikas and Tom Crick and Paola Masuzzo and Anthony Caravaggi and Devin R Berg and Kyle E Niemeyer and Tony Ross-Hellauer and Sara Mannheimer and Lillian Rigling and Daniel S Katz and Bastian Greshake and Josmel Pacheco-Mendoza and Nazeefa Fatima and Marta Poblet and Marios Isaakidis and Dasapta Erwin Irawan and S\'{e}bastien Renaut and Christopher R Madan and Lisa Matthias and Jesper N{\o}rgaard Kj\ae{}r and Daniel Paul O'Donnell and Cameron Neylon and Sarah Kearns and Manojkumar Selvaraju and Julien Colomb},
  Title = {A multi-disciplinary perspective on emergent and future innovations in peer review},
  Journal = {F1000 Research},
  Pages = {1151},
  Volume = {6},
  Year = 2017,
  doi = {10.12688/f1000research.12037.1}
}