Define workflow for paper assignment, communication, and decision making

[tkuhn]

We need to define the specifics of the workflow for paper assignment, communication, and decision making as interactions between the editors-in-chief, the editorial board members, the authors, and the reviewers.

[tkuhn]

Our submission system can take care of much of this process, but it still needs to be configured.

[micheldumontier]

here's some thoughts:

Editors in chiefs

• one editor-in-chief will be made responsible for the submission
• determine whether a submission is in scope to the journal
• identify an editor to manage the submission. This should be done in a balanced manner so as to spread work equally among all editors.
• approve the recommendation and relay the decision to the authors

Editors

• identify 6-8 potential reviewers
• obtain reviews from at least 3 reviewers
• chase requested reviews.
• discuss problematic submissions with the editor-in-chief
• prepare a brief meta-review along with the recommendation (accept, unable to decide, reject)
• submit their recommendation to the editor-in-chief

Reviewers

• Accept or reject a review request in a timely manner (3 days)
• Provide a timely review that largely follows the reviewer guidelines
• Participates in a second review, should one arise
• Provide an ORCID identifier for managing the review request
• Allows their review to be associated with a submission

[tkuhn]

Very good. Then we could have the following process:

• All editors-in-chief are notified about a new submission
• An editor-in-chief is assigned to the submission
• The editor-in-chief performs a quick scan to see whether the paper is not some kind of spam, is written in English, and does not otherwise obviously violate some of the most basic principles
• The editor-in-chief invites an editor for the paper (within at most 2 days after submission); the selection of the editor is based on the areas of expertise and past work load
• If the editor declines the invitation, does not reply within another 2 days, or we receive an out-of-office reply, another editor is invited
• The editor invites 4 or more reviewers (depending on how confident he/she is that they will accept) within at most 2 days
• Reviewers are given another 2 days to accept the invitation and then 10 days to submit their reviews
• If reviewers do not respond or decline, more reviewers are invited to reach a target of 3 reviews
• Reviewers are reminded after 7 days, and again after 9 and 10 days
• Reviewing is concluded when at least 3 reviews are in (2 reviews are acceptable if the result is either a clear Accept or a clear Reject)
• The editor may discuss with the editor-in-chief, in particular for issues that are out of the ordinary
• The editor writes a short meta-review with a recommendation (Accept, Undecided, Reject) within 3 days
• Within at most 2 days, the editor-in-chief processes the recommendation and sends out the respective decision to the authors and makes the reviews public, or discusses with the editor if problems are spotted
• If the decision was Undecided, authors are given up to 60 days to improve their paper
• After resubmission, the process above is repeated with the following two differences: the same editor-in-chief, editor, and reviewers are involved, so nobody needs to be invited (ideally), and the only two possible outcomes are now Accept or Reject (Undecided is only possible for the first round)

That would mean, if all deadlines are exhausted but adhered to, we would have 21 days from submission to the first decision. On the side of the editors-in-chief, we can try to respond within hours whenever possible, to further decrease the needed time.

[RinkeHoekstra]

Although I appreciate the aim for a speedy review process, I do not think we can realistically expect reviewers to provide a review within 10 days. Not for a journal paper.

Also, finding a sufficient number of good reviewers can be very challenging and chances are high that they will not respond within two days (email swamp).

On 8 Dec 2016, at 08:35, Tobias Kuhn notifications@github.com wrote:

Very good. Then we could have the following process:

• All editors-in-chief are notified about a new submission • An editor-in-chief is assigned to the submission • The editor-in-chief performs a quick scan to see whether the paper is not some kind of spam, is written in English, and does not otherwise obviously violate some of the most basic principles • The editor-in-chief invites an editor for the paper (within at most 2 days after submission); the selection of the editor is based on the areas of expertise and past work load • If the editor declines the invitation, does not reply within another 2 days, or we receive an out-of-office reply, another editor is invited • The editor invites 4 or more reviewers (depending on how confident he/she is that they will accept) within at most 2 days • Reviewers are given another 2 days to accept the invitation and then 10 days to submit their reviews • If reviewers do not respond or decline, more reviewers are invited to reach a target of 3 reviews • Reviewers are reminded after 7 days, and again after 9 and 10 days • Reviewing is concluded when at least 3 reviews are in (2 reviews are acceptable if the result is either a clear Accept or a clear Reject) • The editor may discuss with the editor-in-chief, in particular for issues that are out of the ordinary • The editor writes a short meta-review with a recommendation (Accept, Undecided, Reject) within 3 days • Within at most 2 days, the editor-in-chief processes the recommendation and sends out the respective decision to the authors and makes the reviews public, or discusses with the editor if problems are spotted • If the decision was Undecided, authors are given up to 60 days to improve their paper • After resubmission, the process above is repeated with the following two differences: the same editor-in-chief, editor, and reviewers are involved, so nobody needs to be invited (ideally), and the only two possible outcomes are now Accept or Reject (Undecided is only possible for the first round) That would mean, if all deadlines are exhausted but adhered to, we would have 21 days from submission to the first decision. On the side of the editors-in-chief, we can try to respond within hours whenever possible, to further decrease the needed time.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[tkuhn]

Journals like PeerJ and PLOS One also give reviewers just 10 days, and they don't seem to have a problem with that (the same for top journals like Nature and Science, I believe, but they have of course much more leverage).

Personally, when I accept an invitation to review a paper and I am given 30 days, I normally procrastinate and don't do anything for the next 3 weeks, when I receive a reminder and then do the review around the deadline date. If I receive a review request with just 10 days, I often do the review in the next few days. This happened last time with a PeerJ review that I completed in about 2 days. It didn't take more work at all, and was in fact a much nicer experience than having a review request hanging in the air for 30 days. My impression is that reviewers in general complete their reviews shortly before or after the review deadline, no matter how much time they are given.

But would be good to hear other opinions on this.

[micheldumontier]

10 days is the new standard. agree to review, get it done, and keep the ball rolling.