[REVIEW]: OpenSCM Two Layer Model: A Python implementation of the two-layer climate model

whedon commented 3 years ago

Submitting author: @znicholls (Zebedee Nicholls) Repository: https://github.com/openscm/openscm-twolayermodel Version: v0.2.3 Editor: @leouieda Reviewer: @sadielbartholomew, @ashiklom Archive: 10.5281/zenodo.4950772

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/94a3759c9ea117499b90c56421ef4857"><img src="https://joss.theoj.org/papers/94a3759c9ea117499b90c56421ef4857/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/94a3759c9ea117499b90c56421ef4857/status.svg)](https://joss.theoj.org/papers/94a3759c9ea117499b90c56421ef4857)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@sadielbartholomew & @ashiklom, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @leouieda know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @sadielbartholomew

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@znicholls) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @ashiklom

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@znicholls) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

danielskatz commented 3 years ago

👋 @openjournals/dev - see the last comment (References) in https://github.com/openjournals/joss-reviews/issues/2766#issuecomment-792447820 - is there a good answer to this?

znicholls commented 3 years ago

Hi @danielskatz, just to put all the information on the table, I am hoping to do some updates in response to the last review, I am just being slow getting there (I can also do these after the paper is accepted, they're all really helpful suggestions).

arfon commented 3 years ago

👋 @openjournals/dev - see the last comment (References) in #2766 (comment) - is there a good answer to this?

I'm afraid I don't know sorry. Perhaps @tarleb could provide some pointers?

znicholls commented 3 years ago

Progress towards addressing the comments from @sadielbartholomew's excellent review (https://github.com/openjournals/joss-reviews/issues/2766#issuecomment-792447820):

[x] warnings in tests (https://github.com/openscm/openscm-twolayermodel/pull/30)
[x] add dependencies to installation section of the docs (https://github.com/openscm/openscm-twolayermodel/pull/31)
[x] fix docs about running tests (makefile command is wrong) (https://github.com/openscm/openscm-twolayermodel/blob/master/Makefile)
[x] add section in docs about running tests to installation page and in README (https://github.com/openscm/openscm-twolayermodel/pull/30)
[x] fix tests (https://github.com/openscm/openscm-twolayermodel/pull/30)
[x] clearer section on where to get help (https://github.com/openscm/openscm-twolayermodel/pull/30)
[x] re-do the table (https://github.com/openscm/openscm-twolayermodel/pull/29)
[x] briefly compare other models to what is implemented here (https://github.com/openscm/openscm-twolayermodel/pull/29)

@sadielbartholomew please feel free to add any comments etc. on the PRs above. If I don't hear from you in a couple of days, I'll merge them in to keep things moving but we can always revisit later if you still have the desire.

znicholls commented 3 years ago

@whedon generate pdf from branch paper-text-updates

whedon commented 3 years ago

Attempting PDF compilation from custom branch paper-text-updates. Reticulating splines etc...

whedon commented 3 years ago

PDF failed to compile for issue #2766 with the following error:

 Error producing PDF.
! Undefined control sequence.
l.578 Obersteiner, M. (2020). Historical \chem

Looks like we failed to compile the PDF

znicholls commented 3 years ago

@whedon generate pdf from branch paper-text-updates

whedon commented 3 years ago

Attempting PDF compilation from custom branch paper-text-updates. Reticulating splines etc...

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tarleb commented 3 years ago

I'm afraid I don't know sorry. Perhaps @tarleb could provide some pointers?

Looks like a pandoc bug, possibly jgm/citeproc#44. I haven't tried yet, but maybe updating pandoc would fix this?

znicholls commented 3 years ago

Alright I'm all done replying to the second review (https://github.com/openjournals/joss-reviews/issues/2766#issuecomment-792447820), see checklist here https://github.com/openjournals/joss-reviews/issues/2766#issuecomment-826437944 :)

sadielbartholomew commented 3 years ago

Thanks for addressing my comments so thoroughly @znicholls. For confirmation that nothing has inadvertently been changed for the worse when making those improvements I have re-read through the paper and made subsequent basic checks on the codebase (re-running the tests locally, eyeballing some core pages of the current documentation etc.) and everything is great with no issues to report at this stage, except perhaps as a final minor note (I am not sure if you noticed this yourself) that I saw a sentence is now repeated twice in the paper in the main text (lines 33-35) and at the end of the caption for Table 1:

For a more extensive list of simple climate models and references which describe the models in detail, see Table 1 of Nicholls et al. (2020).

and for conciseness you may prefer to remove one instance.

@leouieda as far as I can see both @ashiklom and myself are very happy with this submission and it is ready to go so I believe we can hand it over to you for next steps, etc.?

znicholls commented 3 years ago

for conciseness you may prefer to remove one instance

Great thanks for spotting!

znicholls commented 3 years ago

@whedon generate pdf from branch paper-typos

whedon commented 3 years ago

Attempting PDF compilation from custom branch paper-typos. Reticulating splines etc...

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

arfon commented 3 years ago

👋 @leouieda - is this review about complete now?

leouieda commented 3 years ago

👋🏽 Hi everyone, I'm very sorry for the disappearance. Thank you for continuing on with the review in my absence 🙂

I've gone over the comments and replies and I'm happy to see that all has been properly addressed and both reviewers are happy. I'll do a final proof reading of the paper now and will be happy to move on to publication afterwards.

leouieda commented 3 years ago

@znicholls I see that the latest version of the paper is in the paper-typos branch. I'll my proof-reading on that version. Afterwards, it would be good to merge everything into master.

leouieda commented 3 years ago

@whedon check references

znicholls commented 3 years ago

Awesome thanks

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1175/JCLI-D-12-00195.1 is OK
- 10.1175/JCLI-D-12-00196.1 is OK
- 10.1175/2009JCLI3466.1 is OK
- 10.1002/2015GL064240 is OK
- 10.1007/s10584-011-0156-z is OK
- 10.1007/s00382-019-04686-4 is OK
- 10.5194/gmd-11-2273-2018 is OK
- 10.5194/gmd-13-5175-2020 is OK
- 10.1007/s00382-011-1026-0 is OK
- 10.5194/gmd-8-939-2015 is OK
- 10.5194/acp-11-1417-2011 is OK
- 10.21105/joss.00516 is OK
- 10.5194/bg-17-4075-2020 is OK
- 10.1029/2019gl082887 is OK

MISSING DOIs

- None

INVALID DOIs

- None

znicholls commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

znicholls commented 3 years ago

All merged into master, preview above looks good. Thanks again everyone for your efforts here!

znicholls commented 3 years ago

Oops I forgot to fix my initials

znicholls commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

leouieda commented 3 years ago

@znicholls alright, I'm very happy with the state of the paper and the reviewers are satisfied as well 🎉

There just a few of things that you'll need to dd:

Double check authors and affiliations (including ORCIDs)
Make a release of the software with the latest changes from the review. This is the version that will be used in the JOSS paper. Post the version number here once it's available.
Archive the release on Zenodo/figshare/etc and post the DOI here.
Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

Once those are done, we can move on to publication! Let me know if you need any help with these or have any questions.

znicholls commented 3 years ago

All done :)

[x] Double check authors and affiliations (including ORCIDs) (https://github.com/openscm/openscm-twolayermodel/pull/36)
[x] Make a release of the software with the latest changes from the review. 'v0.2.3'
[x] Archive the release on Zenodo/figshare/etc and post the DOI here. https://doi.org/10.5281/zenodo.4950772
[x] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

znicholls commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

leouieda commented 3 years ago

@whedon set 10.5281/zenodo.4950772 as archive

whedon commented 3 years ago

OK. 10.5281/zenodo.4950772 is the archive.

leouieda commented 3 years ago

@whedon set v0.2.3 as version

whedon commented 3 years ago

OK. v0.2.3 is the version.

leouieda commented 3 years ago

@whedon check references

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1175/JCLI-D-12-00195.1 is OK
- 10.1175/JCLI-D-12-00196.1 is OK
- 10.1175/2009JCLI3466.1 is OK
- 10.1002/2015GL064240 is OK
- 10.1007/s10584-011-0156-z is OK
- 10.1007/s00382-019-04686-4 is OK
- 10.5194/gmd-11-2273-2018 is OK
- 10.5194/gmd-13-5175-2020 is OK
- 10.1007/s00382-011-1026-0 is OK
- 10.5194/gmd-8-939-2015 is OK
- 10.5194/acp-11-1417-2011 is OK
- 10.21105/joss.00516 is OK
- 10.5194/bg-17-4075-2020 is OK
- 10.1029/2019gl082887 is OK

MISSING DOIs

- None

INVALID DOIs

- None

leouieda commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

leouieda commented 3 years ago

@whedon recommend-accept

whedon commented 3 years ago

Attempting dry run of processing paper acceptance...

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1175/JCLI-D-12-00195.1 is OK
- 10.1175/JCLI-D-12-00196.1 is OK
- 10.1175/2009JCLI3466.1 is OK
- 10.1002/2015GL064240 is OK
- 10.1007/s10584-011-0156-z is OK
- 10.1007/s00382-019-04686-4 is OK
- 10.5194/gmd-11-2273-2018 is OK
- 10.5194/gmd-13-5175-2020 is OK
- 10.1007/s00382-011-1026-0 is OK
- 10.5194/gmd-8-939-2015 is OK
- 10.5194/acp-11-1417-2011 is OK
- 10.21105/joss.00516 is OK
- 10.5194/bg-17-4075-2020 is OK
- 10.1029/2019gl082887 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 3 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/2389

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/2389, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

leouieda commented 3 years ago

@znicholls I'm happy to recommend acceptance of this paper for publication in JOSS! 🎉 🎊

Thank you to our reviewers @sadielbartholomew and @ashiklom for your thorough and helpful comments! I'm particularly grateful for all of your patience and understanding with the big delay in this review.

znicholls commented 3 years ago

Awesome thanks @leouieda @sadielbartholomew and @ashiklom ! Much appreciated

danielskatz commented 3 years ago

Thanks @znicholls - this basically looks good to me to finalize, though I've suggested some small changes in the paper in https://github.com/openscm/openscm-twolayermodel/pull/37 Please merge this, or let me know what you disagree with, then we can proceed to publishing.

znicholls commented 3 years ago

Changes look good, all merged

danielskatz commented 3 years ago

@whedon generate pdf

openjournals / joss-reviews