The blog of Valerio Olevano about Scientific Research

A reflection about plagues affecting scientific research

Manifesto to reset the blind peer review system

Manifesto

to reset the blind peer review system

  1. Scientificity - Requested to offer a peer review service, we will limit to check the scientific soundness of the article. That is, we will only check whether the work is really dealing with science or not. E.g., for a manuscript submitted to an astrophysical journal, we will limit to check that the article really deals with astrophysics and not with astrology. Only the latter case can motivate rejection.
  2. Errors - Science historically demonstrated not to be exempt from errors, corrected only after time. Conversely, publications considered in error at short term, could have revealed brilliant intuitions in a future perspective. Consequently, we will refrain to reject publications judged in error by our personal or even collective belief. We will just only limit to signal the possible mistakes to the authors who, as peer and not inferior, could acknowledge them and spontaneously act consequently, or maintain the belief in their correctness and the right to publish.
  3. Fraud - Publication is refused only to manifest fraud supported by strong evidences and incontestable proofs. In suspected frauds, a warning is issued to the authors requiring them to submit their results to a cross-check procedure for reproducibility of their data by referee's own or anyway available and granted facilities.
  4. Quality - It is impossible to assess at short term the future impact of a work, whether it will pave a new avenue or open a new research area, how influential is going to be, whether it is pivotal in solving a problem of subjective criticality or providing a significant contribution, more in general how important, novel, authoritative, or high-quality a work is. Consequently, we will refrain to state any assessment about the level or quality of a work.
  5. Broad Interest - It is difficult to judge how interesting a work in our own field can be to other fields. The broad interest of a paper in one field is better appraised by researchers from other fields. We will assess the broad interest only of works from different fields than ours. By no means the assessment of broad interest for an article necessarily implies high quality or level of the work, they are completely unrelated.
  6. Amendments - Considering sacred the Rights of the Author as stated by the Bern Convention and international copyright agreements, we will refrain to impose any change of any kind to the text of the article prior to publication. We will just limit to suggest possible changes or improvements, leaving intact the right of the author to accept or reject them.
  7. Non-profit publishers - We will offer our peer review services in priority to non-profit non-commercial publishers, academic societies with a historical activity in advancing and diffusing scientific knowledge. Special attention will be offered to green two-way open access publishers, followed by publishers proposing lesser forms of open access.
  8. Lucrative purposes - We will reserve the possibility to not offer peer review services to profit and commercial publishers. In particular, publishers with a dominant commercial position on the market, or with too high margins of profit, or imposing too expensive fares and costs either to authors or to readers.
  9. Seriousness - We will reserve the possibility to not offer peer review services to publishers judged not to provide a sufficiently serious activity in advancing and diffusing scientific knowledge, or publishers recently appeared on the market only attracted by its high margins of profit.

Facsimile of a template report

"The work is scientifically sound and the methodology valid. It is, to the best of my knowledge, original. To the extent I could check, I did not find any issue, mistake, or error that can, bona aut mala fide, affect the findings. I anyway warn the authors that I do not assume any responsibility for possible errors which therefore fall under their full liability.
I solicit the editor to publish the paper."


If you think that the work is not original or contains errors, you anyway accept it without pretending to check the paper again for modifications, and just only signal these points as minor to the editor and for authors check, and leave to the authors the freedom to decide any possible amendment (keeping in mind that we are not co-authors) or the withdrawal of the paper.
In conclusion, a publication can be rejected only in case of: 1) non-scientificity; 2) proved manifest fraud.

If you wish to post a comment or a criticism, contribute to the discussion, tell us your experience, etc. write me (valerio.olevano AT grenoble.cnrs.fr).

Comments:

Feedback from the Deputy Editor of a chemistry review (IF < 3)
After discussing with the editorial board, we agreed that we consider useful a reviewer report that will not focus on the impact and the novelty of the research, if it still provides detailed suggestion to the authors to improve the quality of their presentation and the readability of their paper, so that their science can benefit a broad number of their peer in the scientific community. Thus, we welcome your contribution to review this manuscript. Additionally, despite of XXX being a for-profit organization, I would like to highlight that since more than 50 years ago the YYY journal represents a privileged meeting point where the global community of quantum and theoretical chemists discuss and exchange ideas.

Feedback from the Editor in Chief of a historical very prestigeous review, unfortunately now IF < 3
I don't see the need to 'reset' the peer review system as you suggest (and you can believe us that we have daily frequent experience with it) and Einstein was in my view not opposing it, he just did not like his paper being peer-reviewed at Physical Review when these priciples were just introduced in the 1930s and he hadn't been aware of them yet. However, the principles you have outlined do not sound unreasonable although some of them are a bit difficult to understand so I would suggest a revision of the language. Please also keep in mind that reviewers provide extremely valuable expert recommendations on which basis (plus other information) balanced decisions are being made by editors so this part of the process could be emphasized more strongly in your principles.

Comment by V. Olevano
As you can see, editors of low-IF journals seem to not disdain such a Manifesto. But we must consider that referees are not attracted, do not wish to waste their time with low-IF journals. Consequently, these journals are in short of referees and they are in trouble (their "daily frequent experience"). These editors cannot afford to lose a referee, they are bound to certify as peer-reviewed any article, no matter how this is done, or they will be down-ranked to non peer-reviewed journal. For this they are ready to accept the manifesto, though they would prefer what they consider "the normality", and they try to find excuses for a compatibility of the manifesto with the "normal" principles.
If presenting the manifesto to low-IF journals to probe their aptitude to change and improve the system can be an interesting experiment, in the case of high-IF journals it is better to review articles without announcing in advance the manifesto and the principles which are going to be followed. Indeed, we can expect that also these editors are not disdaining the manifesto, but for another reason. They can use the important information that this referee is accepting any paper, to send her/him only the papers they absolutely want to publish, that is the fashionable papers that are bringing them citations and increasing their IF.