[PRE REVIEW]: NeuralHydrology --- A Python library for Deep Learning research in hydrology

[whedon]

Submitting author: @kratzert (Frederik Kratzert) Repository: https://github.com/neuralhydrology/neuralhydrology Version: v.1.1.0 Editor: @elbeejay Reviewers: @ammilten, @chuckaustin, @jhamman Managing EiC: Arfon Smith

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/d3567e525e20638a8a27c8c6036e1bd7"><img src="https://joss.theoj.org/papers/d3567e525e20638a8a27c8c6036e1bd7/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/d3567e525e20638a8a27c8c6036e1bd7/status.svg)](https://joss.theoj.org/papers/d3567e525e20638a8a27c8c6036e1bd7)

Author instructions

Thanks for submitting your paper to JOSS @kratzert. Currently, there isn't an JOSS editor assigned to your paper.

The author's suggestion for the handling editor is @elbeejay.

@kratzert if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.21 s (653.0 files/s, 80724.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          60           2287           3742           5387
YAML                            17            235            212            759
reStructuredText                53            251            359            666
Jupyter Notebook                 6              0           2704            503
DOS Batch                        1              8              1             26
Markdown                         1              7              0             16
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                           139           2792           7025           7366
-------------------------------------------------------------------------------

Statistical information for the repository '2a2ba3b2b0ac11b51fed224c' was
gathered on 2021/12/21.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Daniel Klotz                     2          1392            437            9.76
Frederik Kratzert               53         12027           2219           76.03
Grey Nearing                     1           259             14            1.46
Martijn Visser                   1             9             10            0.10
Martin Gauch                    30          1582            788           12.65

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Daniel Klotz               1028           73.9          8.7                9.05
Frederik Kratzert          8762           72.9          3.3                9.36
Grey Nearing                199           76.8         10.4                8.54
Martijn Visser                9          100.0          6.7               22.22
Martin Gauch               1418           89.6         11.2                8.39

[whedon]

PDF failed to compile for issue #4020 with the following error:

 Can't find any papers to compile :-(

[kratzert]

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

[arfon]

@whedon generate pdf from branch paper

[whedon]

Attempting PDF compilation from custom branch paper. Reticulating splines etc...

[kratzert]

@whedon generate pdf from branch paper

[whedon]

Attempting PDF compilation from custom branch paper. Reticulating splines etc...

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[kratzert]

Thanks @arfon was just about to do the same. I haven't seen an option to add an alternative branch while submitting the paper.

[elbeejay]

@whedon assign me as editor

[whedon]

OK, the editor is @elbeejay

[elbeejay]

@kratzert I am going to poke around in the repository and the code over the next few days before we sent this out for review, but in the meantime, do you mind suggesting some potential reviewers?

Per the post up top, JOSS does have a list of folks who have indicated they are willing to review submissions, so that may be a good starting point (although that is certainly not a requirement for potential reviewers). When you mention potential reviewers please do so without tagging them with an @ so that we do not unnecessarily ping tons of people. Thanks!

Feel free to reach out if you have any questions, Jay

[kratzert]

Hi @elbeejay

thank you for serving as editor.

Here are some reviewer suggestions from the JOSS reviewer list:

• Andrew Bennett (arbennett): Actually, I think he is/was even working with our library, so he might have some comments/insights from the user perspective already.
• Daniel Buscombe (dbuscombe-usgs): from his fields of expertise, he should be a good match, as he lists hydrology and deep learning
• Raoul Collenteur (raoulcollenteur): hydrologist with good Python skills.

Additional suggestions:

• Joe Hamman (jhamman): One of the xarray dev's and Pangeo contributors and I know he has been using our library as well, so again, maybe a good candidate for feedback from the user/dev perspective.
• Stephan Hoyer (shoyer): Contributer/Dev to various open source projects, such as Xarray, JAX, TensorFlow etc. I have a lot of respect for his software development skills and wanted to suggest him. But since I joined Google recently, we now share the same affiliation which probably eliminates him as a reviewer.

Let me know if I should think of more reviewer candidates and again, thanks for serving as editor.

Enjoy the holidays.

[elbeejay]

@kratzert,

Thank you for the suggestions. I've had a chance to take a look at the package and have a few questions and several suggestions (mostly documentation related) for you before we send this out for review.

• What are the system requirements for someone to train, test, and apply models using this package? I'm guessing a nvidia GPU is required although the documentation does not make this clear. I'd like to clear this question up before reaching out to reviewers as I don't want to get stuck in a situation where we have reviewers who are unable to easily apply and test the package.
• In a similar vein, I think it would be helpful to specify the Python version(s) that are compatible with the neuralhydrology package, via text in the documentation or a requirements.txt file could be good for this
• This should come up in the review process, but I'll note it now, I didn't see any documentation or guidance on how someone would contribute to the package (this is a JOSS review item)
• I think the documentation would be helped by connecting the hosted documentation to the examples/ subdirectory more directly. Right now the existence of those configuration and notebook files is not mentioned in the documentation (as far as I could see), and they could help someone who is trying to work through the tutorials
• As provided, I think those example notebooks themselves could be cleaned up a little to remove "hard paths" to files on developer's local machines in favor of relative paths. Similarly some of the instructions for how to get the data and where to save it such that it meshes with the instructions in the configuration file are not provided in the notebooks themselves.

The key question is really the first one, as that will help us find reviewers who are able to test and review this submission. Subsequent points are not critical at this time, however I will note that clear documentation and easy to run example cases and tutorials often makes the review process go much more smoothly.

Thanks, Jay

[kratzert]

@elbeejay thanks for your quick comments and sorry for my delayed reply:

1) No special system requirements for using our package. Training one of the models on any more or less modern pc/laptop should work and complete within (less than a) minute(s). Sure, training larger models on larger datasets (which we do mostly) benefits heavily from an CUDA-capable GPU but I think that is no requirement for reviewing the package.

2) Good point. We do provide environment files for conda but somehow missed to make this clearer in the documentation. We already have an update in our private development repository that will be pushed to public shortly. For the meantime you can find the environment files here, one for a CPU-only environment and two for different CUDA environments. Minimum Python version should be 3.6, I think. We state 3.7 in our environment files but I think this is just an artifact from when we created those files. I will definitely check what are the minimum requirements.

3) Another good point. We will add a section to the documentation on how to contribute.

4) Again good point, thanks. We will edit the examples to include this information at the very top so that it is clearer from the compiled documentation.

5) Thanks, I will work on that.

I am currently in baby bonding until the second week of January. I won't have much time to work on this until then, since we also have our older son at home all day, because his kindergarten is closed. I hope that is okay.

All the best for the last days of the year.

[elbeejay]

Hi @kratzert this is no problem. The answer to item 1 was really the important piece I wanted to clear up before looking for reviewers.

[elbeejay]

:wave: @dbuscombe-usgs, @jhamman, and @chuckaustin would you be interested in reviewing this submission, "NeuralHydrology --- A Python library for Deep Learning research in hydrology," for JOSS? My understanding is that you all have experience working with either Python or deep learning (or both). At JOSS we have an open checklist-driven review process that takes place through issues on GitHub; review criteria can be found here: https://joss.readthedocs.io/en/latest/review_criteria.html

This is a pre-review issue which we use to find reviewers for a given submission. Once 3 reviewers are found we will formally start the review in a dedicated GitHub issue. At present we are asking reviewers to complete their review of a submission within 6 weeks of the formal review issue opening, although this timeline can be extended as needed.

I'll note that for this submission in particular, I've heard enough about folks in the community using this package that I do not think we will have trouble finding other reviewers if you are unable or uncomfortable with serving as a reviewer. So that being said, please do not feel any pressure to accept this review request; if you are unable to review this and have someone else who you think would be a great reviewer for this package, please mention their GitHub handle here (without the "@" symbol).

Otherwise, if you are interested in reviewing, please just take a look at the JOSS conflict of interest policy before accepting this invitation to ensure we don't run into issues there.

Thank you, feel free to reach out if you have any questions about the JOSS review process. Please do not feel any pressure to accept this review request if you do not have the time or do not feel comfortable reviewing this software package, we appreciate and respect our peer-reviewers' time.

-Jay

[dbuscombe-usgs]

Thanks this looks interesting, but I'm on vacation now and a little slammed when I'm back so I'll skip, but I should be able to help next time

[elbeejay]

@ammilten please confirm that you are able and willing to review this submission for JOSS. Thank you!

[ammilten]

Yes, I can review this submission.

[elbeejay]

@whedon add @ammilten as reviewer

[whedon]

OK, @ammilten is now a reviewer

[elbeejay]

:wave: @jhamman and @chuckaustin just trying to follow-up here. Will be pinging you both via email as well.

@raoulcollenteur would you be at all interested in reviewing this submission? No pressure as I see you've relatively recently reviewed another paper for JOSS.

[chuckaustin]

Hi, yes, I plan to review this submission.

[elbeejay]

@whedon add @chuckaustin as reviewer

[whedon]

OK, @chuckaustin is now a reviewer

[jhamman]

👋 @elbeejay - I'd be happy to review this submission as well. Sign me up!

[elbeejay]

@whedon add @jhamman as reviewer

[whedon]

OK, @jhamman is now a reviewer

[elbeejay]

Awesome. In this case, sorry to have bothered you @raoulcollenteur. We will be transitioning to the review issue so you should not receive any further notifications about this.

[elbeejay]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/4050.