[REVIEW]: agweather-qaqc: An Interactive Python Package for Quality Assurance and Quality Control of Daily Agricultural Weather Data and Calculation of Reference Evapotranspiration

editorialbot commented 8 months ago

Submitting author: @cwdunkerly (Christian Dunkerly) Repository: https://github.com/WSWUP/agweather-qaqc Branch with paper.md (empty if default branch): Version: v1.0.4 Editor: !--editor-->@cheginit<!--end-editor-- Reviewers: @dostuffthatmatters, @dlebauer Archive: 10.5281/zenodo.11122799

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/4a814cae00846c427ed0313749516313"><img src="https://joss.theoj.org/papers/4a814cae00846c427ed0313749516313/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/4a814cae00846c427ed0313749516313/status.svg)](https://joss.theoj.org/papers/4a814cae00846c427ed0313749516313)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dostuffthatmatters & @dlebauer, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @cheginit know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @dostuffthatmatters

📝 Checklist for @dlebauer

editorialbot commented 8 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 8 months ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.10 s (549.9 files/s, 163502.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
HTML                            14            752             42           5074
SVG                              1              0              0           2671
Python                           9            657           1035           2167
JavaScript                      12            131            221            880
CSS                              4            190             35            780
reStructuredText                 6            129             59            266
INI                              2             96              0            212
TeX                              1             21              0            177
Markdown                         2             41              0             86
YAML                             2              1              4             32
-------------------------------------------------------------------------------
SUM:                            53           2018           1396          12345
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 8 months ago

Wordcount for paper.md is 2437

editorialbot commented 8 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1061/9780784408056 is OK

MISSING DOIs

- 10.1002/joc.3413 may be a valid DOI for title: Development of gridded surface meteorological data for ecological applications and modelling
- 10.1061/(asce)0733-9437(1996)122:2(97) may be a valid DOI for title: Assessing integrity of weather data for reference evapotranspiration estimation
- 10.2480/agrmet.64.4.5 may be a valid DOI for title: Quality assessment of weather data and micrometeological flux-impacts on evapotranspiration calculation
- 10.1016/j.agwat.2010.12.015 may be a valid DOI for title: Evapotranspiration information reporting: I. Factors governing measurement accuracy
- 10.1016/j.agwat.2020.106531 may be a valid DOI for title: Conditioning point and gridded weather data under aridity conditions for calculation of reference evapotranspiration
- 10.1016/j.agwat.2020.106376 may be a valid DOI for title: An evaluation of gridded weather data sets for the purpose of estimating reference evapotranspiration in the United States
- 10.1038/srep38495 may be a valid DOI for title: The world’s road to water scarcity: shortage and stress in the 20th century and pathways towards sustainability
- 10.1061/(asce)wr.1943-5452.0001595 may be a valid DOI for title: Validation and Bias Correction of Forecast Reference Evapotranspiration for Agricultural Applications in Nevada
- 10.13031/aea.15325 may be a valid DOI for title: Station Aridity in Weather Monitoring Networks: Evidence from the Oklahoma Mesonet
- 10.1016/s0168-1923(98)00126-9 may be a valid DOI for title: An improved algorithm for estimating incident daily solar radiation from measurements of temperature, humidity, and precipitation

INVALID DOIs

- None

editorialbot commented 8 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

cheginit commented 8 months ago

@dostuffthatmatters & @dlebauer, thanks again for accepting to review this submission. This is the review thread for the submission. All of our communications will happen here from now on.

First, please follow the instructions in the first comment of this thread to create your review checklists.

These checklists contain the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied by editing your checklist comment that our EditorBot will create for you.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#6368 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them, instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2–4 weeks. Please reach out to me if you need more time. We can also use @EditorialBot to set automatic reminders if you know you'll be away for a known period of time.

Please don't hesitate to ping me (@cheginit) with any questions or concerns.

dostuffthatmatters commented 8 months ago

Review checklist for @dostuffthatmatters

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/WSWUP/agweather-qaqc?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@cwdunkerly) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

dlebauer commented 8 months ago

Review checklist for @dlebauer

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/WSWUP/agweather-qaqc?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@cwdunkerly) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

cwdunkerly commented 8 months ago

Thank you @dostuffthatmatters and @dlebauer for agreeing to review. I sincerely look forward to your feedback!

@cheginit I am going to be traveling for training the week of 2/26, and again for field work the week of 3/11. Does more time need to be specifically requested?

cheginit commented 8 months ago

@cwdunkerly It depends on the review goes over the next 4 weeks, i.e., the comments provided by the reviewers and the progress that you made to address them. I will try my best to work with you and accommodate your reasonable request for extension or pause, if needed, while respecting the time and effort of our reviewers.

cwdunkerly commented 8 months ago

I agree 100%. I just have those two weeks, the rest of March is good for me.

cwdunkerly commented 8 months ago

Sorry for the delay in getting these corrections done. I had a combination of reports/work travel/illness these past three weeks.

I am making this a priority for the next week, again I apologize for the delay.

dostuffthatmatters commented 8 months ago

Sorry for the delay in getting these corrections done. I had a combination of reports/work travel/illness these past three weeks.

I am making this a priority for the next week, again I apologize for the delay.

No worries, I also took some time to compile my comments.

I just submitted many of my current review comments. I think the tool is quite helpful and in a good shape. Most of the comments are just about the packaging of it.

cheginit commented 7 months ago

Hi @cwdunkerly, @dlebauer, and @dostuffthatmatters, just checking in on the review progress here.

cwdunkerly commented 7 months ago

@cheginit I have been working on this slowly, however I've been sitting on some of the changes without pushing them.

Let me look over everything and get them pushed. Both @dlebauer and @dostuffthatmatters 's comments have been very clear and helpful.

Thank you for checking in.

cheginit commented 7 months ago

@cwdunkerly Thanks for the update. Hopefully, we'll be able to finish the review in two weeks or so.

cwdunkerly commented 7 months ago

Just reporting in, I am currently working on issue 24, and I have set aside this weekend to complete all outstanding issues that I don't get to during this work week.

cheginit commented 7 months ago

@cwdunkerly Thanks for the update. Sounds good.

cwdunkerly commented 7 months ago

The last issue outstanding is #19 which pertains to paper text corrections. I'm resolving this tomorrow, I'm just conferring with the other authors first.

Once this is done, what is the next step? The only other issue I am aware of is adding the other installation methods (pipenv, PDM) to the docs on the github.io page, which I will also have resolved tomorrow.

If I have missed issues detailed somewhere else I apologize.

dostuffthatmatters commented 7 months ago

I only have to parse the bibliography and check for any non-cited sources/comparable packages. I will try to get that done by tomorrow.

cheginit commented 7 months ago

@cwdunkerly We should wait for @dostuffthatmatters and @dlebauer to check all their review boxes and let me know that they're done with their review and there's no outstanding blocking issue. Afterward, I will do a final check and begin the publication process.

cheginit commented 7 months ago

@dostuffthatmatters Thank you!

cwdunkerly commented 7 months ago

The last thing I have to do is make higher res versions of the figures. I'll have that done once I get out of my meetings this morning.

Do I need to reupload the paper anywhere when done or does JOSS pull it from the repo?

cheginit commented 7 months ago

@cwdunkerly You can make all the necessary changes to your paper.md file, figures, and references in your repo, commit them, then use editorialbot to generate a new PDF (@editorialbot generate pdf).

cwdunkerly commented 7 months ago

@editorialbot generate pdf

editorialbot commented 7 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

cheginit commented 7 months ago

Hi @dlebauer, just a friendly reminder about the review.

dlebauer commented 7 months ago

@cwdunkerly this is a nice paper and I appreciate all of the improvements that you have made in response to feedback. Well done, almost there! The only thing remaining is to make sure that data, code, and instructions to reproduce the figure 2 are provided https://github.com/WSWUP/agweather-qaqc/issues/28

dostuffthatmatters commented 7 months ago

I have the same impression as @dlebauer. The tool is well done and the repository has improved a lot in the last weeks. I only see two things from my end:

[x] I opened a PR because there was a lot of metadata that could be added to the bibliography: https://github.com/WSWUP/agweather-qaqc/pull/29
[x] The installation instructions should be updated to the ones from the README

After that, I recommend it to be accepted. Good work, @cwdunkerly!

cwdunkerly commented 7 months ago

Thank you both very much for your supportive comments. I do have one question:

@dlebauer I had used the CIMIS weather station located in Davis, CA for my example files as it's a very well documented station, but the data used for figure 2 is from a station in Reno, NV. Would it be satisfactory if I was to include this Reno station as "example_data_3" and include text in the figure description detailing which file it came from?

@cheginit I am out of the office until Tuesday morning, I will have these final comments resolved and pushed that day, and I'll send a message when it's done.

cheginit commented 7 months ago

@cwdunkerly Sounds good!

dlebauer commented 7 months ago

Would it be satisfactory if I was to include this Reno station as "example_data_3" and include text in the figure description detailing which file it came from?

Outside of a JOSS submission I think that would be sufficient. But given my understanding of JOSS’ reproducibility requirement, I think the data is insufficient, there should be code and instructions for reproducing the figure. This could be in a simple md file alongside the dataset.

cwdunkerly commented 6 months ago

I was delayed in getting back to this, I will be taking care of it tomorrow and over the weekend if necessary.

Update: I've got just a bit more to do, it'll be done Monday.

Update 2: Unfortunately I lost a tire during my commute today, some scrap metal in the road shredded the tire pretty bad. I'm unharmed but dealing with this has delayed me. I apologize and I will get this done as soon as possible.

cheginit commented 6 months ago

@cwdunkerly Sounds good.

cwdunkerly commented 6 months ago

I have completed all outstanding tasks, sorry for the delay. If I have missed something please let me know.

@dostuffthatmatters I have updated the docs with installation instructions, and I changed the language for the contributor roles in the paper, if you wished to look over that as well.

@dlebauer I have added instructions for re-creating figure 2, as requested under 28. If you think more is needed please let me know.

@cheginit the final thing I wish to do is review the paper one final time later today. I'm catching a flight this afternoon and I plan to do that then.

cwdunkerly commented 6 months ago

@editorialbot generate pdf

editorialbot commented 6 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

cheginit commented 6 months ago

Update 2: Unfortunately I lost a tire during my commute today, some scrap metal in the road shredded the tire pretty bad. I'm unharmed but dealing with this has delayed me. I apologize and I will get this done as soon as possible.

@cwdunkerly Sorry to hear about your accident, and I am glad to hear that you're safe. Please let me know if you need more time.

cwdunkerly commented 6 months ago

@editorialbot generate pdf

editorialbot commented 6 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

cwdunkerly commented 6 months ago

Other than @dlebauer (didn't mean to ping again so soon, sorry.) confirming whether 28 is sufficiently resolved, I have two minor questions regarding the paper:

For line 153 in the article proof, is "top 10 percentile" or "top 10th percentile" more grammatically correct? The paper is currently "10th". I changed it after editorialbot generated the paper this most recent time.
Richard Allen is the only name in the in-paper citations that gets spelled out or has initials included. Rick is a fantastic guy and he's been a great teacher/mentor, but it does feel weird to me that the only citations like this happen to be for a name on the paper. Is there an easy way to change this beyond editing the bibliography directly?

Thanks.

dlebauer commented 6 months ago

Looks great, thanks!

I think “10th percentile“

cheginit commented 6 months ago

@dlebauer and @dostuffthatmatters Thanks for the time and effort that you put into reviewing this submission and help improving it!

@cwdunkerly I noticed that Richard Allen has three different spellings: Richard Allen, Richard G. Allen, and R. G. Allen. Please modify your bib file and also his name in the author list to match the name in all his other publications (which it seems to be Richard G. Allen).

cheginit commented 6 months ago

@editorialbot check references

editorialbot commented 6 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1002/joc.3413 is OK
- 10.1061/(asce)0733-9437(1996)122:2(97) is OK
- 10.2480/agrmet.64.4.5 is OK
- 10.1016/j.agwat.2020.106531 is OK
- 10.1016/j.agwat.2020.106376 is OK
- 10.13140/RG.2.2.35811.60967 is OK
- 10.13140/RG.2.2.21605.88808 is OK
- 10.1038/srep38495 is OK
- 10.1061/(asce)wr.1943-5452.0001595 is OK
- 10.1111/1752-1688.12956 is OK
- 10.13031/aea.15325 is OK
- 10.1016/s0168-1923(98)00126-9 is OK
- 10.1061/9780784408056 is OK

MISSING DOIs

- No DOI given, and none found for title: Crop evapotranspiration-Guidelines for computing c...
- No DOI given, and none found for title: Agricultural Evapotranspiration and Net Irrigation...
- No DOI given, and none found for title: Agricultural Evapotranspiration and Net Irrigation...
- No DOI given, and none found for title: Agricultural Evapotranspiration and Net Irrigation...
- No DOI given, and none found for title: Coping with water scarcity: an action framework fo...
- No DOI given, and none found for title: ASCE Standardized Reference Evapotranspiration (ET...
- No DOI given, and none found for title: Bokeh: Python library for interactive visualizatio...
- No DOI given, and none found for title: REF-ET: Reference evapotranspiration calculation s...

INVALID DOIs

- None

cheginit commented 6 months ago

@cwdunkerly Also, please add ORCID of other authors if they have one. There are several references without any DOI, and it appears that they don't have one. Please confirm the missing DOIs.

cheginit commented 6 months ago

@cwdunkerly Once you're done with addressing my previous two comments, please do the following, so I can then move forward with recommending acceptance of the submission:

[ ] Make a tagged release of your software, and list the version tag of the archived version here.
[ ] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository). Make sure that the license listed for the archive is the same as the software license!
[ ] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[ ] Please list the DOI of the archived version here.

cheginit commented 6 months ago

@cwdunkerly A friendly reminder to finish the tasks above.

cwdunkerly commented 6 months ago

@cheginit Sorry for the delay, I had several field trips last week. I believe I have done everything you asked, to recap:

Author names have been standardized across references.
I have added ORCiDs for the co-authors that have one.
I have confirmed that the references with missing DOIs do not have them.
- I have added URL's to the bibliography for two sources without DOIs.
I have made a tagged release of the software (v1.0.4)
I have archived the software (in the form of a .zip of the above tagged release) to Zenodo.
- The software license matches the github repo
- The metadata matches the paper draft
- The DOI is 10.5281/zenodo.11122799

cwdunkerly commented 6 months ago

@editorialbot generate pdf

openjournals / joss-reviews