[REVIEW]: Pychron: Automated Data Collection and Reduction for Noble Gas and Ar/Ar Geochronology

whedon commented 3 years ago

Submitting author: !--author-handle-->@jirhiker@Kevin-Mattheus-Moerman<!--end-editor-- Reviewers: @pvermees, @JennaEverard Archive: Pending

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/a857f7037497c8b6c6f377f777a89296"><img src="https://joss.theoj.org/papers/a857f7037497c8b6c6f377f777a89296/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/a857f7037497c8b6c6f377f777a89296/status.svg)](https://joss.theoj.org/papers/a857f7037497c8b6c6f377f777a89296)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bovee & @pvermees, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @richardjgowers know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @bovee

Conflict of interest

[ ] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@jirhiker) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @pvermees

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@jirhiker) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bovee, @pvermees it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

PDF failed to compile for issue #3386 with the following error:

 Can't find any papers to compile :-(

arfon commented 3 years ago

@bovee, @pvermees – This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above.

Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention https://github.com/openjournals/joss-reviews/issues/3386 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

arfon commented 3 years ago

@whedon generate pdf from branch manuscript

whedon commented 3 years ago

Attempting PDF compilation from custom branch manuscript. Reticulating splines etc...

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=5.96 s (382.1 files/s, 58251.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                        2072          56215          79593         173004
Fortran 77                      13            611            722           8551
C                                2            820           2477           7068
Fortran 90                       8            348            572           4949
reStructuredText               127           1599           3356           3099
YAML                            19             37             31           1484
Markdown                        12            199              0            784
Bourne Shell                     6             82             45            321
Arduino Sketch                   7            130            113            287
make                             1             24              5            124
C++                              1             39              2            122
MATLAB                           1             19             12             54
Cython                           2             11             23             41
C/C++ Header                     1             14              1             37
INI                              1             12              0             37
Mako                             1              7              0             15
DOS Batch                        1              1              0              3
HTML                             1              0              0              1
CSS                              1              0              8              0
-------------------------------------------------------------------------------
SUM:                          2277          60168          86960         199981
-------------------------------------------------------------------------------

Statistical information for the repository '9efc82ae0a247482cbf822ca' was
gathered on 2021/06/19.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
AGES-NGX                         6           143             59            0.02
Argon                           22           828            384            0.11
ArgonServer                      1            19             14            0.00
Bryan Wathen                     1            28              6            0.00
GSFC-MNGRL                       1            85              2            0.01
Jake Ross                     3794        319103         245878           53.36
Jan                              6           143             79            0.02
Stephen                          3           219              4            0.02
Stephen E Cox                   69           605            401            0.10
ael-ng                           3            29             24            0.01
ages-ngx-admin                  13           776            379            0.11
angl_laser                       2            84             49            0.01
argonlab2                       61          1283            761            0.19
argonlab5                        6           104             64            0.02
argus1ms                       106          2592           1548            0.39
felix                          172          8860           4775            1.29
jake                             1             2              2            0.00
jake ross                       93          1950            952            0.27
jan                              2            37             13            0.00
jirhiker                      1849        350877          96180           42.22
jross                            3             8             34            0.00
mcintosh                        33           428            231            0.06
nmgrluser                      164          5317           3459            0.83
obama                           60          2883           1913            0.45
raregas                         14           458            197            0.06
sidneyhemming                    1             5              2            0.00
stephen-e-cox                   44          1667            723            0.23
usgshelix                        1            25             23            0.00
uv                              30          1318            757            0.20
wiscar                           2            17              7            0.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
AGES-NGX                     83           58.0         38.0                0.00
ArgonServer                   2           10.5         58.9                0.00
Bryan Wathen                 10           35.7         34.6                0.00
GSFC-MNGRL                    2            2.4         13.4                0.00
Jake Ross                313175           98.1         36.2               23.60
Stephen E Cox               740          122.3         26.2               15.41
ael-ng                        7           24.1          4.4                0.00
ages-ngx-admin              462           59.5         35.7                5.63
angl_laser                   82           97.6         13.2                6.10
argonlab2                   220           17.1         15.1               10.45
argonlab5                    33           31.7         40.8               12.12
felix                      4196           47.4         40.2               19.04
mcintosh                    108           25.2         53.5                8.33
nmgrluser                  2447           46.0         30.9               11.36
raregas                     321           70.1         13.3                9.03
sidneyhemming                 1           20.0         34.2                0.00
usgshelix                     9           36.0         30.9                0.00
uv                           18            1.4         86.1               27.78
wiscar                       17          100.0         30.3                0.00

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

bovee commented 3 years ago

Hi, sorry I don't check my Github notifications very often. I barely might have been able to do this six months ago, but I don't have any capacity for this review now. Thanks for understanding.

whedon commented 3 years ago

:wave: @pvermees, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @bovee, please update us on how your review is going (this is an automated reminder).

richardjgowers commented 3 years ago

@whedon remove @bovee as reviewer

whedon commented 3 years ago

OK, @bovee is no longer a reviewer

danielskatz commented 3 years ago

👋 @richardjgowers - Can you provide an update on this submission and review?

richardjgowers commented 3 years ago

@danielskatz I've had trouble finding reviewers for this paper, partly due to the specific hardware required, so it's currently at a stage where only one person has reviewed it

danielskatz commented 3 years ago

What's the next step? If we can't find another reviewer, we may have to mark it as withdrawn. But in any case, we shouldn't just leave it hanging.

richardjgowers commented 3 years ago

I’ll try to find another reviewer, perhaps someone new is available now.

On Thu, Sep 30, 2021 at 19:47, Daniel S. Katz @.***> wrote:

What's the next step? If we can't find another reviewer, we may have to mark it as withdrawn. But in any case, we shouldn't just leave it hanging.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/3386#issuecomment-931575392, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGSGB6DJPIRIG5X3XLWAKLUESWDTANCNFSM4662QIWQ .

jirhiker commented 3 years ago

I can suggest a few more reviewers. Ones that have pychron installed and are using it daily. @aschaen @bjicha @stephen-e-cox @sidneyhemming @cmmercer @wrguenthner

arfon commented 2 years ago

@aschaen @bjicha @stephen-e-cox @sidneyhemming @cmmercer @wrguenthner – would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html

The submission under consideration is Pychron: Automated Data Collection and Reduction for Noble Gas and Ar/Ar Geochronology

stephen-e-cox commented 2 years ago

There's a fundamental problem here that you guys have already noted: no one outside of our user group has the hardware necessary to test this software, but none of us are really independent reviewers. I think all of us as users have probably made a (comparatively very small) number of contributions to the code, have regularly corresponded with Jake about new features and troubleshooting, and are both financially and intellectually invested in this project. But I think it's important that our field move beyond leaving the software we use every day languishing in the grey literature, so I'm happy to write a review it if the journal is ok with all of that.

We also have a new user at Columbia, @jennaeverard, who has been setting up a new pychron-controlled analytical setup in our lab as part of her senior thesis project. If she's willing to consider contributing a review, I think it would be valuable because she has done the setup pretty recently with minimal input from me and I believe none directly from Jake. But she's also a college student working on her senior thesis and might not have the time to take this on right now.

Kevin-Mattheus-Moerman commented 1 year ago

@stephen-e-cox thanks for sharing that. I understand regarding the conflict of interest information.

@JennaEverard, if you could help that would be greatly appreciated.

Kevin-Mattheus-Moerman commented 1 year ago

@richardjgowers can you resume the search for reviewers here?

Kevin-Mattheus-Moerman commented 1 year ago

@aschaen @bjicha @sidneyhemming @cmmercer @wrguenthner – would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html

The submission under consideration is Pychron: Automated Data Collection and Reduction for Noble Gas and Ar/Ar Geochronology

JennaEverard commented 1 year ago

@Kevin-Mattheus-Moerman Sure, I'd be happy to contribute a review.

Kevin-Mattheus-Moerman commented 1 year ago

@editorialbot add @JennaEverard as reviewer

editorialbot commented 1 year ago

@JennaEverard added to the reviewers list!

Kevin-Mattheus-Moerman commented 1 year ago

@editorialbot remove @richardjgowers as editor

editorialbot commented 1 year ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

Kevin-Mattheus-Moerman commented 1 year ago

@editorialbot assign me as editor

editorialbot commented 1 year ago

Assigned! @Kevin-Mattheus-Moerman is now the editor

Kevin-Mattheus-Moerman commented 1 year ago

@jirhiker We will proceed to review now. Apologies for all the delays encountered here!

Kevin-Mattheus-Moerman commented 1 year ago

@editorialbot start review

editorialbot commented 1 year ago

Can't start a review when the review has already started

Kevin-Mattheus-Moerman commented 1 year ago

@JennaEverard :wave: you can get started with your review. Thanks for your help! To get started you can generate your reviewer checklist by using: @editorialbot generate my checklist. Let me know if you have any questions.

JennaEverard commented 1 year ago

Review checklist for @JennaEverard

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/NMGRL/pychron?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@jirhiker) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Kevin-Mattheus-Moerman commented 1 year ago

@pvermees, @JennaEverard thanks for your help with this review. I hope you are getting on well. Let me know if you have any updates to report. Thanks.

pvermees commented 1 year ago

I can't tick the following check box in my review: Installation: Does installation proceed as outlined in the documentation?, because I still haven't managed to install PyChron on my system (Ubuntu 22.04). Consequently, I cannot complete the remainder of the review either.

The Docker image throws the following error:

Could not connect to any X display.
ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['python', './pychron/launchers/launcher.py']' command failed.  (See above for error)

The main GitHub repo (NMGRL/pychron) tells me to install a command line utility called pcm. Following the instructions (pip install pcm followed by pcm wizard), I get:

pcm wizard
Usage: pcm [OPTIONS] COMMAND [ARGS]...
Try 'pcm --help' for help.

Following the link to https://github.com/PychronLabsLLC/pcm, and reinstalling pcm by entering pip install pychron-cm; then trying pcm wizard again, I get lots of errors, the last of which is:

FileNotFoundError: [Errno 2] No such file or directory: 'edm'

Installed edm, and got some more luck with pcm wizard, although I still get the following warnings:

No candidate found for requirement 'keyring'
Did you mean this?
    pyparsing

I don't know how to proceed next. The documentation (which dates from 2019) is intimidating and unclear https://pychron.readthedocs.io/en/latest/user_guide/setup/quick_start.html. It is unclear how the "Quick Start (ish)" section relates to a code base that is installed through pip.

jirhiker commented 1 year ago

@pvermees @Kevin-Mattheus-Moerman

NMGRL/pychron_docker has been update and tested on Ubuntu 22.04 and macOS 12.1
,3. ,4. pcm is a separate project from PychronLabsLLC that shouldn't be considered for this review
Installation and configuration of pychron is almost always contracted out hence the lack of good open installation documentation. Pychron_docker was the compromise to let users use a "test" version of pychron.

@pvermees https://github.com/NMGRL/pychron_docker

curl -O https://raw.githubusercontent.com/NMGRL/pychron_docker/main/pychron.sh
chmod +x ./pychron.sh
./pychron.sh

This worked for me on macOS and Ubuntu 22.04. FYI On Ubuntu i did have an issue where new windows were black but resizing the window caused it to properly refresh.

pvermees commented 1 year ago

@jirhiker Docker works now, so I've ticked the Installation check box in the review form. Can you share some example Peak Hop files so that I can test (part of) the functionality? Alternative/additionally, can you direct me to your unit tests (if you have them)?

jirhiker commented 1 year ago

@pvermees Peak hop can only be used with a data collection version.

The unittests for peakhop (https://github.com/NMGRL/pychron/blob/dev/dc/pychron/test_suite.py, https://github.com/NMGRL/pychron/blob/dev/dc/pychron/experiment/tests/peak_hop_parse.py) are only for testing parsing of the peak hop files.

In general, the test_suite runs automatically via a Github Action, its coverage is relatively low but I do try to have it cover the core calculations.

Some example pyscripts are located at https://github.com/NMGRL/pychron/tree/dev/dc/docs/user_guide/operation/scripts/examples

Kevin-Mattheus-Moerman commented 1 year ago

@pvermees, @JennaEverard thanks again for your help with this review. Could you provide an update on review progress? Thanks!

pvermees commented 1 year ago

I have checked the installation box, but am unable to go further than that. As @jirhiker explained:

Installation and configuration of pychron is almost always contracted out hence the lack of good open installation documentation.

You would really need to ask an actual user to get feedback on these aspects of the software.

Kevin-Mattheus-Moerman commented 1 year ago

@JennaEverard Could you provide an update on review progress? Thanks!

JennaEverard commented 1 year ago

@Kevin-Mattheus-Moerman I am working on the Functionality portion of my review and aiming to have it completed by the end of this week. I will provide an update on Friday! (Update: I will finalize my review by Tuesday, May 2nd - I just need to finish the functionality section)

Kevin-Mattheus-Moerman commented 1 year ago

@JennaEverard can you provide an update please? Thanks.

Kevin-Mattheus-Moerman commented 1 year ago

@JennaEverard can you provide an update please? Thanks.

JennaEverard commented 1 year ago

@Kevin-Mattheus-Moerman I have completed my review!

Kevin-Mattheus-Moerman commented 1 year ago

@JennaEverard thanks. Would you be happy to tick all the boxes :point_up: and to formally recommend acceptance? Thanks

Kevin-Mattheus-Moerman commented 1 year ago

@JennaEverard :point_up:

jirhiker commented 1 year ago

@Kevin-Mattheus-Moerman @JennaEverard Anything I can do to move this along

Kevin-Mattheus-Moerman commented 1 year ago

@jirhiker apologies for the delays encountered with this submission. Currently we have hit an obstacle as one reviewer cannot complete this review. In fact it has come to light that people outside of your research group cannot fully use/install/evaluate the software as it stands. I have just paused this submission because I am currently consulting with the editorial board, since this fact makes this work unacceptable for JOSS.

openjournals / joss-reviews