Better guidelines for article content

[kyleniemeyer]

In keeping with some recent ideas of having a "Guides" page (for authors, reviewers, and editors), rather than putting everything in the "About" page, I think it might be useful to include a bit more in the author guide, particularly for article content.

I don't think we should accept articles that only includes two or three sentences, or a single paragraph. (The example on the "About" page is fairly short, which perhaps contributes to our more minimal submissions.) Submissions can be around a page and still require minimal effort to compile.

If we want others to take JOSS articles seriously, then I think we (the editors) need to decide the minimum content needed for an article.

[kyleniemeyer]

(I'm not suggesting that we move to longer articles, just something more than the few sentences/single paragraph of some submissions that we've accepted.)

[labarba]

What would be a reasonable minimum length? 250 words? And maximum? 1000 words?

[kyleniemeyer]

Sure, that sort of range seems reasonable, and would help avoid the criticism of JOSS articles not being "articles."

I think we could incorporate new guidance/examples (possibly supplemented by good examples of articles already published) into a new "Guides" page, that also includes things like how we recommend citing the version used in research vs. citing the JOSS article (both, typically), links to setting up repositories with Zenodo integration, etc.

[labarba]

"The "Summary" section of the article (in the paper.md file) should be between 250 and 1000 words, and include a general description of the software functionality, and a statement of need. The latter should make clear what is the research application of the software. Utilities of general use that do not focus on a scholarly research function are not within the scope of JOSS. We also recommend that you include mention—if applicable—of ongoing research projects using the software or recent scholarly publications enabled by it."

[labarba]

Also, maybe it's not a bad idea to say what we mean by "research software." Too many folks submitting out-of-scope stuff recently.

"JOSS publishes articles about research software. This definition includes software that: solves complex modeling problems in a scientific context (physics, mathematics, biology, medicine, social science, neuroscience, engineering); supports the functioning of research instruments or the execution of research experiments; extracts knowledge from large data sets; offers a mathematical library, or similar."

[labarba]

And third, we might add a sentence like:

"JOSS welcomes submissions from broadly diverse research applications. For this reason, we request that authors include in the paper some sentences that would explain the software functionality and domain of use to a non-specialist reader."

I'm finding that submissions are often loaded in jargon, and you have no clue what the software might be for or about, unless you're in the field. As an editor, it makes it hard to identify reviewers. As a JOSS reader, it's uninviting and might discourage you to submit your own work.

[kyleniemeyer]

Yes, absolutely. I think it's really important than articles start with a general, understandable description of the domain and problem.

[labarba]

Who can get these recommendations into the website? Does @arfon need to do it?

Certainly, the EiC needs to approve any changes like this. But maybe he's expecting a pull request to suggest them. I personally am not going to mess with the website on raw HTML source files. It also feels like overkill to have to clone the whole JOSS repo just to suggest some edits to the author guide.

[kyleniemeyer]

I can add this via a pull request—I've generally been doing it by editing the files on GitHub via the web interface, rather than cloning and editing on my computer.

[labarba]

Ah, good point.

[arfon]

I can add this via a pull request—I've generally been doing it by editing the files on GitHub via the web interface, rather than cloning and editing on my computer.

I think we should start with a pull request here and comment further on the text there. One thing that's been mentioned in the past is that the example paper.md is too short (and not a real paper).

Perhaps we should use a more complete (real) example for a paper such as @cMadan's one from 10.21105/joss.00031

[lheagy]

We might consider encouraging authors to include a short # Statement of need (or# Purpose) section in their article. Having this be explicit and distinct from the summary would help clarify scope and elevate the importance of this statement.

[wojdyr]

I like all the suggestions above. One minor note. I'd rather omit ongoing and recent the sentence:

We also recommend that you include mention—if applicable—of ongoing research projects using the software or recent scholarly publications enabled by it.

I imagine that examples of projects/papers are a big help in ensuring that the software has a research function. Perhaps it could be a required part of the submission form?

On a related note, one journal I'm familiar with, J. Appl. Cryst., has a section "Computer Programs" with a requirement in the author guidelines:

The utility of the program and adequacy of the documentation should normally have been proved by the successful use of the program by at least two independent users.

This is may too strict requirement for JOSS, but it is a reasonable way to make sure that the program is useful not only to the author.

[kyleniemeyer]

@wojdyr: thanks for the feedback!

Regarding the last bit, the peer reviewer(s) are expected to install and run the software, and ensure it works as described (including any examples/test cases)

[arfon]

@kyleniemeyer - just wanted to check you're planning on submitting some modifications in the form of a PR for us to consider?

[kyleniemeyer]

@arfon yep, on my todo list, but I won't be able to work on this until Tuesday at the earliest, after a proposal is submitted.

[arfon]

👍 thanks @kyleniemeyer