search

openjournals / paper-JOSS-oneyear

Paper describing design and first-year of JOSS
Creative Commons Attribution 4.0 International
0 stars 0 forks source link

PeerJ R3C3 #26

Closed kyleniemeyer closed 6 years ago

kyleniemeyer commented 6 years ago
  1. The review process primarily uses 1 reviewer - what is the confidence that this is sufficient? If the review process is primarily checklist based (as appears) with little judgement, then this could be enough.
kyleniemeyer commented 6 years ago

See #18

karthik commented 6 years ago

The assessment above, and Arfon's comment on 18 seem sufficient to address this.

arfon commented 6 years ago

I have added the same response as #18 to this:

The editor-in-chief (Arfon Smith) who initially created JOSS has a background in astrophysics where a single reviewer is standard however we recognize that this is not normal for many research fields. Second reviews are possible, and some editors regularly solicit multiple reviews for the JOSS submissions they are handling, and we are planning on building first-class support for multiple reviewers into our infrastructure.

We would also like to point out that the goal of the review is to bring the paper to an acceptable standard, not to judge it for acceptance or rejection. Since this is the case, it's not clear that any more than one reviewer is needed to provide this help and feedback, particularly if we can say that the editors are closely involved in checking the work of the reviewer.

danielskatz commented 6 years ago

slight grammar changes:

The editor-in-chief (Arfon Smith) who initially created JOSS has a background in astrophysics, where a single reviewer is standard. However, we recognize that this is not normal for many research fields. Second reviews are possible, and some editors regularly solicit multiple reviews for the JOSS submissions they are handling, particularly when the editor is less confident about their own understanding of the area, software, and review. We plan to build first-class support for multiple reviewers into our infrastructure.

We would also like to point out that the goal of the review is to bring the paper to an acceptable standard, not to just to judge it for acceptance or rejection. Since this is the case, it's not clear that any more than one reviewer is needed to provide this help and feedback, particularly if we can say that the editors are closely involved in checking the work of the reviewer.