Review process

[filippo82]

Check out the review process of Solid Earth which is published by EGU.

[leouieda]

I've been through the review process at an EGU journal a few years ago. To be honest, apart from being open there wasn't anything different. The conversation was still not real time (they review, we reply) unlike JOSS. We could probably do a better job of recruiting comments from the community as well (through slack and the swung Twitter).

One thing I really didn't like is there whole thing with the separate Discussion publication. When you submit, they publish your paper in a " Discussion" variant of the journal which gets it's own doi. Upon acceptance, the paper is published in the "" journal with a different DOI. The problem is that this confused the hell out of Google Scholar et al and people trying to cite your paper.

[filippo82]

I've been through the review process at an EGU journal a few years ago. To be honest, apart from being open there wasn't anything different. The conversation was still not real time (they review, we reply) unlike JOSS.

I can see that, for the Software part of the hub mission, it would be great to have comments in real time from reviewers. However, for the manuscript part, I am not sure whether this would work or not. Usually, reviewers might need to go through the text multiple times and then modify their comments. Also, some comments might make sense only when the whole review is ready.

We could probably do a better job of recruiting comments from the community as well (through slack and the swung Twitter).

This suggestion is definitely something that I am not used to! The idea of getting comments through these unconventional channels would need accurate regulation. Often, Editors have enough trouble making decisions when they need to take into account the responses of three reviewers. I could imagine that this would add complexity to the editorial process. Maybe this could be limited to the software part of the submission.

One thing I really didn't like is there whole thing with the separate Discussion publication. When you submit, they publish your paper in a " Discussion" variant of the journal which gets it's own doi. Upon acceptance, the paper is published in the "" journal with a different DOI. The problem is that this confused the hell out of Google Scholar et al and people trying to cite your paper.

I wasn't aware of this and I totally agree with you.

[leouieda]

I could imagine that this would add complexity to the editorial process.

You're absolutely right. Maybe this could be a good way to get post-publication review?

Usually, reviewers might need to go through the text multiple times and then modify their comments. Also, some comments might make sense only when the whole review is ready.

Again, spot on. By real time, I don't mean that the reviewers should post small comments about the manuscript. I mean doing away with the traditional "stages" of the review process. Why should the authors have to wait for the editor to gather all comments? Why should there be only 1-2 rounds of review? Why have rounds in the first place? I always feel a bit frustrated when I want to ask a reviewer for clarification but there is no easy to way to communicate.

Of course, as we venture into this we need to be especially careful of bias and harassment (see #5). The 1 advantage I see in the traditional model is that the editor can serve as a line of defense between authors and reviewers. Granted, this is not usually successful. I think we'll need to have a very clear Code of Conduct and be strick about enforcing it.

[joferkington]

It may be worth a brief tangent: We should be open to changing the review process if whatever model we start with doesn't work well.

"Better peer review" is a really tough problem, and I doubt anyone can get it right at present, no matter how much thought and discussion goes into the first iteration. I think it's important to be willing to change the review process, particularly early on.

In summary, I'd like to see a statement that the process itself is open to review and will change over time. I think it's important to:

• Have a process
• Document the process
• Iterate on the process
• Document the changes

Of course this still doesn't answer the key question: "What should the first iteration look like?".

[geodancer]

Thank you for your invitation for comments. The following are my thoughts about the review process that I have held close to the vest for several years.

Once upon a time, I was an Associate Editor of Geophysics. I hated doing it because the articles were almost always unreadable. I mean, consider your college lab reports. They weren't meant to be read by anyone. They were for letting the professor know you dotted the i's and crossed your t's in lab. Your report demonstrated you did the work and got the expected outcome. I suggest pretty much every journal submission is to let the world know the author did something. Truly teaching the theory and method are not part of the manuscript.

Scientific American is an amazing magazine because their sizeable editorial staff massages articles about complex topics into readability. The papers are frequently invited, I suspect from docile authors who will accept the heavy massaging. When I say "readable" I mean, something that explains and teaches. If you think about it, those are high demands. In my profession, exploration geophysics, I think the only journal that does this is CSEG's Recorder. They are willing to give many pages to an author to overcome the terse message of a column of equations.

Let me highlight my key words this way ... The usual submission is a demonstration, mostly to the author, that he or she did something. Most submissions neither explain nor teach. When I say "explain," I mean, in the best possible way, when you ask someone to listen with an open mind, you say, "Let me explain," and then you lead the listener, in small steps, into why your idea has merit. When you say, "Let me explain," you are working to communicate to your listener. In the journals I read, I think this is rare.

Now for the double-scary part. This proposed journal will also have code. Will it teach? That is, will it be delightfully commented or will it be the equivalent of a column of numbered equations?

Why do our journals publish college lab reports instead of something that explains and teaches? Possibly any of the following: (1) Reviewers have been steeped too long in this BS and consider it normal. (2) Reviewers are experiencing "imposter syndrome" so they can't resist the tide. (The author is the expert. What do I really know?) (3) Everyone is on a clock. There is no time allotted to the several rounds of editing that this would require. This would be way out of normal for the editor and the author.

Where do you set the bar for this new journal?