Ask the experts. What do journal editors think on peer review?

Peer Review Week 2016 is nearing to the end. Following my previous post containing guidelines for reviewers by Dafydd Gibbon I present wider discussion on the current state of peer review by editors of De Gruyter Open journals.

The main concern raised by the editors is that constantly growth of whole academic publishing, triggers growth in number of papers that need reviewers’ work. At the same time number of good reviewers is limited.

Ugo Gianazza, Editor-in-Chief of Open Mathematics says:

It gets more and more difficult to find reviewers. If the number of journals and the number of submitted manuscripts keep growing, we cannot rule out the possibility that the system crashes.

Under this point of view I wonder whether the present situation is a direct consequence of the way academia assesses research work. By this I mean the classical “publish or perish”, together with all the numerology connected with impact factor, h-index, number of citations per person, etc.

Francesco Simoni, editor of Optical Data Processing and Storage points out that facing problems mentioned by Gianazza, the current review system occurs to be outdated:

The present peer review system should be completely changed.

This system was born on old publishing landscape when the journals number for each topic was limited and specialization was not so narrow as at present. The result is that experts are often overwhelmed by review requests which they are not interested to perform. The result is that they either refuse or do it just by giving a quick look at the paper with a comment (either positive or negative) of low quality. Only young researchers are willing to perform the review since it is a new experience for them, but they have usually not enough knowledge to perform the review based on the actual state of the art of the analyzed topic. Then it is very difficult to find reviewers, the reviews are often not satisfactory, there is a long delay in the publication process.

The solution of the above mentioned problems could be find if the publisher would consider the reviewing process among the costs of publication. One possibility for the journal would be appointing a number of experts as reviewers (depending of the type of journal could be in the range of tens), by paying them a little amount for each review performed.

In fact it is hard to say will money be a good incentive for overworked experts to write reviews. The very same researchers are not paid by journals for authoring their articles, yet they are very eager to write them. The reason for that are rules of academic funding and promotion. Number of papers published is a main indicator used to evaluate their work by their supervisors and funders. At the same time reviewing works of the others is not seen as a important input to research by funders, despite the fact that peer review is still as a crucial part of publication process. This asymmetry in research work evaluation criteria seems to be a main source of the problem.

Section Editor of Open Chemistry, Laszlo Peter:

The community of potential reviewers does not expand at the same rate as the number of submissions. In order to maintain the good level of the papers published, the compensation of reviewers by some means should be considered. This would also spin up the rate at which reviews can be obtained. Paying reviewers might help, but I am not absolutely sure it would. A professional recognition system where the number of reviews and the quality of the reviews counts similarly to the publications – I think this might help.

Publons is a try to create such professional recognition system. However it is still quite a novel idea and is not widely used.

Hans R. Herren, Editor-in-Chief Open Agriculture:

Although based on a reciprocity and well meaning model, the present peer review system needs to include some form of reward to continue to operate at the expected quality level, given that more and more scientists are under time and funding pressure, with the dwindling of public funding and the growing of private funding, where the deliverables leave little free time for voluntary work. It could be money or credits. The needs for reviewers differ, so the reward system should be flexible.

Peter Crosthwaite, section editor at Open Linguistics points out also the problem of time:

The process takes too long, but I’m not really sure about how to solve that. It might be better to have a central pool of professional in-house reviewers whose job it is solely to review papers, so as to speed up the time considerably – if they were paid a small fee for this, it might speed up the process.

However, relaying on in-house review only, in time of increasing academic specialization, would result in my opinion, in high chance that a paper will be reviewed by somebody with a little knowledge on its actual subject. Yet, growing number of papers that need to be reviewed results anyway in an increased need for pre-screening, which is done in-house.

Miguel Yus, Open Chemistry Editor-in-Chief:

Due to the increasing number of scientific journals and consequently the huge number of manuscripts to be evaluated, I think that every journal should have a high-scientific level team to make a preselection/preevaluation of any received paper before sending it to real experts.

What a review should contain?

Another issue with peer review pointed out by several editors is insufficient standardization.

Hanns Mohammer, Editor of Biomonitoring journal:

I feel it is often not clear for the reviewers what their role should be: should they help improve the paper or should they rather serve as gate keepers and help to reduce the increasing amount of published papers? How should they deal with poor language? Is it their job to improve it or should they just ask for a clearer version before they undertake their task?

Peter Crosthwaite expands these issues:

Reviewers themselves often criticize papers for things outside of the author’s control, basically suggesting a study that the reviewer would like to see based on their personal beliefs or ideals, rather than constructively looking at the conditions surrounding a particular paper and offering advice regarding how to solve the issues with that paper in that context. Not every paper can end up in nature or applied linguistics​ – at some point, reviewers should focus on what can be done to make the paper ‘work’ with a view to one day accepting it, rather than just dismissing the paper out of hand.

Reviewers focus too much on minor language issues – this should be the copy editor’s job. I get really mad when the reviewer says things like ‘A native speaker should check the data’, which is both insulting to me as an author, and borderline racist for authors from non-native backgrounds.

Eugenijus Butkus, Section Editor at Open Chemistry concludes:

A more structured peer review form would facilitate a review process for reviewers.

Hans Mohammer added also two interesting and more general points:

The electronic systems were a great improvement for publishing. But often the software is not really helpful for reviewers, and it makes their works harder (e.g. by need of logging-in to do their work).

As an author I often felt that reviewers have not read or understood my paper correctly. As a reviewer I often had the impression that the authors do not really respond to my suggestions. As an editor I learned to despair about both authors and reviewers. Most often a more polite way of communication would help a lot!

Well knonwn concerns about the real author’s anonymity in so called double blind review were raised by Ugo Gianazza:

As far as double-blind peer review is concerned, I think it is almost a fairy tale: when I finish a manuscript, I post it on arXiv, and I update my web-page, with the corresponding link. This is a very important way to let the community know about my work, and also get comments. Therefore, even if my name were canceled from a paper, a reviewer would get to know who the author is.

This is echoed by Editor-in-Chief of Open Lingusitcs, Sabine Ehrhart, who says:

Blind peer review does not work since we know each other by our ideas and not only by names.

The most positive view on the current system was presented by Anna Maria Fino from mathematical journal Complex Manifolds:

The current peer review system is working well. For our journal (and in general for journals in Mathematics) it is very important to do the review before the publication (after the publication it does not make sense). Moreover, I think that for most of the cases it is sufficient to send the paper to one referee instead of two referees, one can use two referees just in case in doubts.

Fuente: <http://openscience.com/>