[REVIEW]: DIFFICE-jax: Differentiable neural-network solver for data assimilation of ice shelves in JAX

openjournals / joss-reviews

Reviews for the Journal of Open Source Software

Creative Commons Zero v1.0 Universal

725 stars 38 forks source link

[REVIEW]: DIFFICE-jax: Differentiable neural-network solver for data assimilation of ice shelves in JAX #7254

Open editorialbot opened 2 months ago

editorialbot commented 2 months ago

Submitting author: !--author-handle-->@wangyji@AnjaliSandip<!--end-editor-- Reviewers: @daniel-cheng, @nmcardoso, @RahulSundar Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/4cf986d53809937f2352b5fc2b7a6120"><img src="https://joss.theoj.org/papers/4cf986d53809937f2352b5fc2b7a6120/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/4cf986d53809937f2352b5fc2b7a6120/status.svg)](https://joss.theoj.org/papers/4cf986d53809937f2352b5fc2b7a6120)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@daniel-cheng & @nmcardoso & @RahulSundar, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @AnjaliSandip know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @daniel-cheng

📝 Checklist for @nmcardoso

📝 Checklist for @RahulSundar

editorialbot commented 2 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 2 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.04 s (1153.2 files/s, 212526.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          21            491            777           1510
TeX                              2            247              0           1456
Jupyter Notebook                 5              0            960           1273
Markdown                        10            232              0            634
YAML                             2              8             10             45
TOML                             1              6              0             34
MATLAB                           1             12             13             32
-------------------------------------------------------------------------------
SUM:                            42            996           1760           4984
-------------------------------------------------------------------------------

Commit count by author:

   616  yjwang
   111  Yao Lai
     6  Yongi Wang

editorialbot commented 2 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.5194/tc-15-1975-2021 is OK
- 10.1002/2014JF003181 is OK
- 10.1029/2024JH000169 is OK
- 10.1017/jog.2020.112 is OK
- 10.1016/j.jcp.2023.112428 is OK
- 10.5194/gmd-16-6671-2023 is OK
- 10.1017/jog.2021.120 is OK
- 10.1017/jog.2022.41 is OK
- 10.1017/jog.2023.73 is OK
- 10.1016/j.jcp.2024.112865 is OK
- 10.21203/rs.3.rs-2135795/v1 is OK
- 10.1029/JB094iB04p04071 is OK
- 10.3189/S0022143000015744 is OK
- 10.1038/s43247-022-00385-x is OK
- 10.1073/pnas.2309788121 is OK
- 10.5194/tc-9-1427-2015 is OK
- 10.1002/jgrf.20125 is OK
- 10.5194/tc-5-315-2011 is OK
- 10.3189/172756403781816338 is OK
- 10.1137/19M1274067 is OK
- 10.48550/arXiv.2207.02338 is OK
- 10.1016/j.jcp.2018.10.045 is OK
- 10.1029/2021MS002621 is OK
- 10.1017/jog.2023.8 is OK
- 10.1029/2010GL043853 is OK
- 10.1016/j.jcp.2023.112435 is OK
- 10.4208/cicp.oa-2020-0164 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Deep learning the flow law of Antarctic Ice Shelve...
- No DOI given, and none found for title: MEaSUREs Phase-Based Antarctica Ice Velocity Map, ...
- No DOI given, and none found for title: MEaSUREs BedMachine Antarctica, Version 2
- No DOI given, and none found for title: Euler operators for mis-specified physics-informed...
- No DOI given, and none found for title: JAX: composable transformations of Python+NumPy pr...

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None

editorialbot commented 2 months ago

Paper file info:

📄 Wordcount for paper.md is 1315

✅ The paper includes a Statement of need section

editorialbot commented 2 months ago

License info:

✅ License found: MIT License (Valid open source OSI approved license)

editorialbot commented 2 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

nmcardoso commented 2 months ago

Review checklist for @nmcardoso

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/YaoGroup/DIFFICE_jax?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@wangyji) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

AnjaliSandip commented 1 month ago

@daniel-cheng & @RahulSundar, here's a friendly reminder to begin reviewing this software/paper.

AnjaliSandip commented 1 month ago

@nmcardoso How close are you to completing the review? @daniel-cheng & @RahulSundar, do you have time to review this submission in the next 2-3 weeks?

RahulSundar commented 1 month ago

Hi, I have been working on the review. Will be able to complete within next two weeks.

On Mon, 21 Oct, 2024, 9:45 pm Anjali Sandip, @.***> wrote:

@nmcardoso https://github.com/nmcardoso How close are you to completing the review? @daniel-cheng https://github.com/daniel-cheng & @RahulSundar https://github.com/RahulSundar, do you have time to review this submission in the next 2-3 weeks?

— Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/7254#issuecomment-2427128317, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFZGIWREY4XRKYSOGP36O6LZ4USCJAVCNFSM6AAAAABOSIVACGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRXGEZDQMZRG4 . You are receiving this because you were mentioned.Message ID: @.***>

daniel-cheng commented 1 month ago

Hi @AnjaliSandip, I will also target having the review done within the next two weeks. Thanks for your understanding!

daniel-cheng commented 1 month ago

Review checklist for @daniel-cheng

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/YaoGroup/DIFFICE_jax?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@wangyji) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1. Contribute to the software 2. Report issues or problems with the software 3. Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

AnjaliSandip commented 1 week ago

@nmcardoso, @daniel-cheng & @RahulSundar, how close are you to completing the review?

RahulSundar commented 1 week ago

Review checklist for @RahulSundar

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/YaoGroup/DIFFICE_jax?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@wangyji) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1. Contribute to the software 2. Report issues or problems with the software 3. Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

RahulSundar commented 1 week ago

Dear Author @wangyji,

Here are some of my comments. General feedback: The work and its application to Ice sheet modelling is novel and the code base helps tackle inverse problems of relevant. The paper is well written and crisp. The code base is easily installable without much fuss. The tutorials, examples are well explained. The paper would be a good contribution!

Specific comments:

Kindly include - automated unit tests for the code base in a separate directory called tests.
Include a statement of need in the README.md file
Would have been nice to have a sphinx documentation for the code rather than the markdown files.
Please include detailed contribution guidelines if possible instead of just the contact email address.

Hope most of these would be possible to implement.

RahulSundar commented 1 week ago

@nmcardoso, @daniel-cheng & @RahulSundar, how close are you to completing the review?

HI @AnjaliSandip ! Thanks for the reminder. I needed to check their examples and tutorials one by one which took a while. I have completed my review and have shared my comments as well. Regards, Rahul

daniel-cheng commented 1 week ago

Hi @AnjaliSandip, I am wrapping up my review within the next two days, but am actively working on it now. I will update as soon as I am done with the evaluation. Thanks again for the reminder rand understanding!

daniel-cheng commented 5 days ago

Hi @wangjyi, I've been able to complete the review checklist and evaluated this submission.

General Comments

This submission provides a new way to perform data assimilation and inverse modeling of ice shelves using Physically Informed Neural Networks (PINNs). The codebase is fully functional, and the provided tutorials/examples serve to document as well as reproduce the results of the submission. The manuscript is well written, and the submission as a whole is well put together, fulfilling most major checklist requirements. After addressing a few technical details noted below, I can recommend acceptance at the editor’s discretion.

Specific Comments

There is an unresolved reference on Page 2 Line 45 (?; ?).
While the tutorial/examples provide a series of manual tests to run in a way, some automated tests could be included. These could be extensions of existing examples, comparing generated results against archived results and ensuring it falls below some threshold.
Similarly, perhaps the inclusion of pretrained model weights along with results for comparison would aid with validation and automated testing as well as user experience
Documentation - while present in the form of Readmes covering the files purposes/concepts, and docstrings in the functions themselves - could be made more accessible. Consider an automated documentation generator to scrape and condense the existing docstrings into a more accessible format.