search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
722 stars 38 forks source link

[REVIEW]: Tethys: a Spatiotemporal Downscaling Model for Global Water Demand #5855

Closed editorialbot closed 5 months ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@ifthompson<!--end-author-handle-- (Isaac Thompson) Repository: https://github.com/JGCRI/tethys Branch with paper.md (empty if default branch): main Version: v2.1.0 Editor: !--editor-->@kthyng<!--end-editor-- Reviewers: @Mariosmsk, @nickrsan Archive: 10.5281/zenodo.10966693

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/3e2b44b441ec718b995ddcbd0cfa3149"><img src="https://joss.theoj.org/papers/3e2b44b441ec718b995ddcbd0cfa3149/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/3e2b44b441ec718b995ddcbd0cfa3149/status.svg)](https://joss.theoj.org/papers/3e2b44b441ec718b995ddcbd0cfa3149)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@Mariosmsk & @nickrsan, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @jsta know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @nickrsan

📝 Checklist for @Mariosmsk

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5194/hess-22-2117-2018 is OK
- 10.5334/jors.197 is OK
- 10.1038/s41597-023-02086-2 is OK
- 10.5194/hess-17-4555-2013 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 1 year ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.03 s (1471.6 files/s, 104552.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          19            380            269            769
SVG                              4              4              4            298
CSS                              2             54             11            244
reStructuredText                 8            227            235            220
YAML                             4             24             14            109
TeX                              1              0              0             52
Markdown                         2             18              0             48
DOS Batch                        1              8              1             26
JSON                             1              0              0             20
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            43            719            541           1795
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 561

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jsta commented 1 year ago

Hello @nickrsan and @Mariosmsk! Thanks so much for agreeing to review. Links to reviewing information are in the top message. You can generate your checklist with @editorialbot generate my checklist. Please let me know if you have questions.

nickrsan commented 1 year ago

Review checklist for @nickrsan

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

nickrsan commented 1 year ago

@jsta Thank you! I'll work through this when I have time, but likely not for a few weeks as mentioned before (but on my last review found it helpful to make the checklist early so I could knock out small parts when I have time). Looking forward to taking a look at the package

Mariosmsk commented 1 year ago

Review checklist for @Mariosmsk

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

nickrsan commented 1 year ago

Hi all, just checking in to say I'm finally at a point where I can schedule some time to conduct this review and start filling in my checklist. Thanks for your patience.

Mariosmsk commented 1 year ago

Hi @ifthompson,

Some comments so far.

License: Does the repository contain a plain-text LICENSE file with the contents of an [OSI approved] (https://opensource.org/licenses/alphabetical) software license?

I recommend updating the License file to include an OSI-approved software license. Additionally, please ensure that the setup.py file correctly references the chosen license. This will help ensure compliance with open-source licensing standards.

Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.

I checked the usage example, I have the image after spatial downscaling but not the image before. It's essential to also provide the original image before downscaling.

Installation: Does installation proceed as outlined in the documentation?

Installation proceeded as outlined in the documentation. However, I recommend adding matplotlib to the requirements.txt file to ensure all dependencies are clearly documented and can be easily installed. Regarding the Dask example, I encountered Java-related errors, even though I believe I have added Java to the path. For better reproducibility, consider including code in the example that automatically configures the Java path. This will improve the user experience and minimize potential issues during execution.

kthyng commented 11 months ago

Hi @nickrsan and @Mariosmsk! How are your reviews coming along?

@ifthompson Have you been able to respond to all of the reviewer comments?

nickrsan commented 11 months ago

Hi there - I apologize for my slowness, but it hasn't fallen off my radar. I'll schedule time for this next week. Thanks for your patience!

kthyng commented 11 months ago

@nickrsan Great!

@Mariosmsk @ifthompson How are things on your end?

nickrsan commented 11 months ago

Just making a note for the license checkbox since GitHub isn't picking up the license from its format and advertising it by name - it appears to be a 2-clause BSD license - not anything the authors need to respond to, just to mark this for the review.

nickrsan commented 11 months ago

Hi all,

As a progress update, I've filed a few issues - mostly minor, but blocking further review, on the tethys repository (see cross-references above from GitHub). Also, I'll note that two unit tests are currently failing on my machine (Win 11, Python 3.10) - though I think one is that I probably need to set the JAVA path (test_regional_query) and the other I'll dig into and file an issue if appropriate (test_easy_query not producing expected output).

kthyng commented 10 months ago

Hi @ifthompson — we haven't heard from you for awhile. Still with us?

jsta commented 10 months ago

@crvernon Do you have any insight into Issac's availability that might inform this review?

crvernon commented 10 months ago

@jsta Let me see if I can reach him. If not, I will handle any reviewer comments. Thanks!

kthyng commented 9 months ago

I'm going to take over as editor here!

@crvernon Have you been able to get in contact with the author? If not can you take over? If progress can't be made on this soon, I'd like to pause it for now, or withdraw if it looks like the submission won't be able to be finished.

kthyng commented 9 months ago

@editorialbot assign me as editor

editorialbot commented 9 months ago

Assigned! @kthyng is now the editor

crvernon commented 9 months ago

@kthyng I am going to go ahead and take this one over as the author point of contact as I have not been able to get in touch with the lead author. Please forward any other requests to me. I'll get fully caught up on this thread later today and respond in full. Thanks!

kthyng commented 9 months ago

@crvernon is the author list changing order or do you want to take over as the listed author in this issue?

crvernon commented 9 months ago

@kthyng I'll be the point of contact on this but let's leave Isaac as the designated author. I will add myself as a shared first author and will adjust the paper accordingly.

@nickrsan and @Mariosmsk thank you for your patience! I will be taking over this review so please reach out if you have any questions or issues. I have caught up on the issues you have raised and will begin to systematically address those. I'll link all activity to this thread so we can have a single point of access to all of our conversations.

kthyng commented 8 months ago

Ok great I'll check in next week if I don't hear from any of you in the meantime.

kthyng commented 8 months ago

@crvernon Just checking in. What's your status?

crvernon commented 8 months ago

@kthyng going well. Just knocking out a few more things in the backlog that the reviewers had identified. Should have the other PRs merged in this week with a summary of what is left, if anything, to move forward. Thanks!

crvernon commented 8 months ago

Working out some final things...have had a busy week. I'll get the comments knocked out ASAP. Thanks!

crvernon commented 8 months ago

Coming off of business travel this week. Next week is looking good! Apologies for the delay!

kthyng commented 7 months ago

@crvernon I would suggest we add a "paused" label to reflect a delay in this work (otherwise no impact to the submission). If it will take longer than 1 or 2 months, it might be better to withdraw and resubmit. What do you think?

crvernon commented 7 months ago

@kthyng if I cannot get it done by the end of the week let's pause as you suggest. I'll post here at the end of the week either way. Thanks!

crvernon commented 7 months ago

:wave: @Mariosmsk

Addressing your comments...all fixes are now available on main. I have release a new version (2.1) with the following changes accounted for as well. Thank you!

1) --------------------------------------------------

License: Does the repository contain a plain-text LICENSE file with the contents of an [OSI approved] (https://opensource.org/licenses/alphabetical) software license?

I recommend updating the License file to include an OSI-approved software license. Additionally, please ensure that the setup.py file correctly references the chosen license. This will help ensure compliance with open-source licensing standards.

No problem. I reformatted the BSD 2 Simplified license to be recognized by GitHub and also added in the correct SPDX short identifier in setup.py. This was done in https://github.com/JGCRI/tethys/pull/65

2) --------------------------------------------------

Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.

I checked the usage example, I have the image after spatial downscaling but not the image before. It's essential to also provide the original image before downscaling.

The data that gets downscaled is from the input CSV file. You can see this in the example data and it gets referenced in the configuration file that is used to run tethys. This tabular data gets allocated spatially to this raster in the example (also in the example data): data/maps/regions.tif.

Once you have downloaded the example data via tethys.get_example_data() you can run the following to see what is actually in the example configuration file (or you can open it up in an editor):

import tethys

# assuming you downloaded to the default location
config_file = tethys.default_download_dir + '/example/config_example.yml'

with open(config_file) as cfg:
    print(cfg.read())

this prints...

# Basic example config file for Tethys

# Project level settings
years: [2010, 2015, 2020, 2025, 2030]
resolution: 0.125
demand_type: withdrawals

# output settings
output_file: output/example_output.nc

# csv input
csv: data/example_data.csv

# dictionary of proxy files and promised variables and years
proxy_files:
  data/population/ssp2_{year}.tif:
    variables: Population
    years: [2010, 2020, 2030]

# mapping of inputs sectors to proxies
downscaling_rules:
  Municipal: Population

# list of map files
map_files:
  - data/maps/regions.tif

You can see the input CSV file which holds the data to be downscaled in csv and the raster containing the regions as the first item in map_files.

All of this information is available when you run tethys via the variable you store your run initialization in. For example:

# run tethys using the config file
result = tethys.run_model(config_file)

You can use result to get anything in the configuration, like the csv path:

result.csv

returns:

'data/example_data.csv'

Based on your comment, I decided to build plotting functionality within the package that allows you to produce an interactive visualization the input data to be downscaled with the regional spatial data. So you can now do the following using the result object from your Tethys run:

from tethys import InputViewer

# instantiate input viewer
gridded_inputs = InputViewer(
    result,
    sector="Municipal",
    year=2020,
)

# plot interactive figure
gridded_inputs.plot(cnorm="log")

Of course, you can also call help on this method to see the options available:

help(gridded_inputs.plot)

I also added in a method to allow the dataset to be exported to a raster file, so:

gridded_inputs.to_raster("<your raster path>")

This was done in https://github.com/JGCRI/tethys/pull/68

3) --------------------------------------------------

Installation: Does installation proceed as outlined in the documentation?

Installation proceeded as outlined in the documentation. However, I recommend adding matplotlib to the requirements.txt file to ensure all dependencies are clearly documented and can be easily installed. Regarding the Dask example, I encountered Java-related errors, even though I believe I have added Java to the path. For better reproducibility, consider including code in the example that automatically configures the Java path. This will improve the user experience and minimize potential issues during execution.

I completely agree concerning the package dependencies. This was fixed in https://github.com/JGCRI/tethys/pull/68 For the Java path configuration, I chose to place a notice to the user above that example section. These types of path configurations can vary based on OS and package manager utilization and can sometimes cause more grief to the user. But thank you so much for highlighting this as a need! This was done in https://github.com/JGCRI/tethys/pull/69

crvernon commented 7 months ago

:wave: @nickrsan - some of your comments were addressed in the above response, but the others that were specifically addressed where done in the following:

Thanks so much for these great comments!

crvernon commented 7 months ago

:wave: @kthyng, @Mariosmsk, @nickrsan

I believe I have addressed all comments in the review. Thanks and let me know if you have any further questions!

crvernon commented 6 months ago

:wave: Hi @Mariosmsk and @nickrsan just checking to see if I addressed all of your concerns and if you were all good on your side of things? Thanks so much for your patience throughout this review from our side of things!

Mariosmsk commented 6 months ago

Hi @crvernon, it's good from my side, seems okay! Thank you!!

crvernon commented 6 months ago

Thanks @Mariosmsk!

kthyng commented 6 months ago

@Mariosmsk If you're satisfied can you check off all the boxes in your review checklist and make sure any issues you're opened are closed, if they are finished (some may be appropriate to leave open).

@nickrsan Are there any outstanding issues from your review?

nickrsan commented 6 months ago

Hi all, thanks for the updates - I haven't been able to schedule time to continue the review yet, but will see what time I can schedule this week.

kthyng commented 6 months ago

@nickrsan Ok thanks for the note, I'll wait to hear from you.

crvernon commented 6 months ago

:wave: @nickrsan just circling back to see if my recent updates based on your review were satisfactory? Thanks!

nickrsan commented 6 months ago

Thanks for the ping. Working on it right now and will update by end of day

nickrsan commented 6 months ago

OK, this looks great and like a nicely polished package. I've checked off my checklist and think it's ready to go.

I'll note that I had one issue, which is that I couldn't get the Dask example to run, but could run the main example. Tracking it down a bit more (by trying to run the example_demeter.yml file without the dask config, as well as running the test suite), I get the impression that maybe there's an issue in the gcamreader dependency - java is set up on this machine and available on the path - Java is throwing some errors when being executed though. The failure being somewhere in gcamreader is suggested to me by the test_regional/test_load_region_data test failing on my machine too. I'd consider this to not affect the core claims of the package though, but wanted to flag this for you. I could provide more information in an issue on the tethys repo if you think it's something you want to track down.

crvernon commented 6 months ago

Thanks @nickrsan! I'll note your issue in the gcamreader repository and have someone check that out. Thanks!

Also, thanks for your review and feedback. Much appreciated!

@kthyng I think we are good to go now.

kthyng commented 5 months ago

Ok then it's that time for the post-review checklist @crvernon!

kthyng commented 5 months ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance

kthyng commented 5 months ago

@crvernon let me know when you're done and I'll take over from there.

crvernon commented 5 months ago

@editorialbot set v2.1.0 as version

editorialbot commented 5 months ago

Done! version is now v2.1.0