search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
725 stars 38 forks source link

[PRE REVIEW]: globalemu: A flexible framework for emulating the sky-averaged 21-cm signal from the cosmic dawn and epoch of reionization #3181

Closed whedon closed 3 years ago

whedon commented 3 years ago

Submitting author: @htjb (Harry Bevins) Repository: https://github.com/htjb/globalemu Version: v1.1.1 Editor: Pending Reviewer: Pending Managing EiC: Kyle Niemeyer

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @htjb. Currently, there isn't an JOSS editor assigned to your paper.

The author's suggestion for the handling editor is @dfm.

@htjb if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands
whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.26 s (105.5 files/s, 10564.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          15            261            260            850
TeX                              1             16              0            234
reStructuredText                 5            141            198            187
Markdown                         1             17              0            106
YAML                             2              8              8             45
Jupyter Notebook                 1              0            276             41
DOS Batch                        1              8              1             26
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            27            455            750           1498
-------------------------------------------------------------------------------

Statistical information for the repository '97b6124c18fcb9bace74111e' was
gathered on 2021/04/17.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Harry Bevins                    82          1563           1064           51.98
htjb                            21          1567            860           48.02

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Harry Bevins               1206           77.2          1.8                5.39
whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/nature11177 is OK
- 10.1093/mnras/stu1744 is OK
- 10.1093/mnras/stx2065 is OK
- 10.3847/1538-4357/ab07be is OK
- 10.1093/mnras/staa1530 is OK
- 10.1038/nature25792 is OK
- 10.1038/s41586-018-0796-5 is OK
- 10.1093/mnras/stz3388 is OK
- 10.3847/1538-4357/ab2879 is OK
- 10.1088/1538-3873/ab5bfd is OK
- 10.1093/mnras/staa3091 is OK
- 10.1109/ICEAA.2019.8879199 is OK
- 10.1093/mnras/stab152 is OK

MISSING DOIs

- None

INVALID DOIs

- None
kyleniemeyer commented 3 years ago

@whedon query scope

whedon commented 3 years ago

Submission flagged for editorial review.

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

kyleniemeyer commented 3 years ago

Hi @htjb, thanks for your interest in JOSS. Because of the relatively small size of the codebase, the JOSS editorial board is going to review your submission to see if it meets our requirement for substantial scholarly effort.

htjb commented 3 years ago

Hi @kyleniemeyer, great to hear from you! If it helps your decision;

danielskatz commented 3 years ago

@htjb - I'm sorry to say that after discussion amongst the JOSS editors, we have decided that this submission does not meet the substantial scholarly effort criterion for review by JOSS. Please see https://joss.readthedocs.io/en/latest/submitting.html#other-venues-for-reviewing-and-publishing-software-packages for other suggestions for how you might receive credit for your work.

danielskatz commented 3 years ago

@whedon reject

whedon commented 3 years ago

Paper rejected.

htjb commented 3 years ago

@danielskatz - Thanks for getting back to me on this. I am disappointed that you will not consider this for publication. Could you please clarify for my future reference which of the 'substantially scholarly effort' criterion the software does not meet? Thanks!

danielskatz commented 3 years ago

The size of the code is small - anything under 1000 LOCs is a question for the editors.

In addition, some editors noticed that there is already an astronomy paper about this result and that the paper is more about the result than the software. And that the code does not appear to be as professionally engineered as it could: some routines save data files on disk as a side affect, many routines lack a docstring, etc.

If the code is expanded in the future, and more polished, we would welcome a resubmission.

htjb commented 3 years ago

@danielskatz Thanks for the feedback.

In addition, some editors noticed that there is already an astronomy paper about this result and that the paper is more about the result than the software.

Just to clarify, I did highlight, for transparency, the MNRAS preprint paper in my submission to JOSS and the result is merely used to illustrate the capabilities of the software. The point is that the software represents a significant improvement on previous work (which incidentally has been heavily cited see my previous comment) and it can be easily retrained by a user on new sets of models.

And that the code does not appear to be as professionally engineered as it could: some routines save data files on disk as a side affect, many routines lack a docstring, etc.

Everything that is saved is saved as part of the trained instance of the network into one file that can later be evaluated in nested sampling loops for example. I take the point about the docstrings and this is something I will resolve. However, everything that is intended for use by the user is well documented and this has not previously been an issue with publishing in JOSS for me.

Thanks again for the feedback anyway it is appreciated. I think you will miss out on some potential citations personally!