[REVIEW]: FrEDI: An R package for estimating future climate impacts within the United States

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@knoiva-indecon@pdebuyl<!--end-editor-- Reviewers: @tonyewong, @hahsan1 Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/a7c382ac5257ff28de021514a4388101"><img src="https://joss.theoj.org/papers/a7c382ac5257ff28de021514a4388101/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/a7c382ac5257ff28de021514a4388101/status.svg)](https://joss.theoj.org/papers/a7c382ac5257ff28de021514a4388101)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@tonyewong & @hahsan1, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @pdebuyl know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @tonyewong

📝 Checklist for @hahsan1

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 1 year ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.32 s (352.2 files/s, 107294.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                       6           2100           1931           7038
HTML                            35           1188            104           6681
R                               40           1129           3830           6157
Rmd                             12            848           1905            681
Markdown                         9            146              0            379
CSS                              3             54             10            230
YAML                             5             20             15            127
XML                              1              0              0             84
TeX                              1              5              0             54
SVG                              1              0              1             11
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                           114           5490           7796          21443
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 1 year ago

Wordcount for paper.md is 1478

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5194/egusphere-2023-114 is OK
- 10.1073/pnas.1517056113 is OK
- 10.1007/s10584-021-03048-6 is OK

MISSING DOIs

- 10.1163/9789004322714_cclc_2021-0166-513 may be a valid DOI for title: Climate Change and Social Vulnerability in the United States: A Focus on Six Impacts

INVALID DOIs

- None

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

pdebuyl commented 1 year ago

@tonyewong @hahsan1 please check the information text at the top of this page.

As the review takes place into this discussion page, feel free to provide a partial review at some point or to ask clarifications to the author here.

Some reviewers also open issues at the repository under review. This is fine for bug reports or requests about the documentation, for instance. In that case, please report the resulting bugfix or other result from the discussion here so that I can get proper notification.

tonyewong commented 1 year ago

Review checklist for @tonyewong

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/USEPA/FrEDI/?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@knoiva-indecon) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

tonyewong commented 1 year ago

@pdebuyl I reread the COI policy more carefully as I began my review. Dr. Hartin and I collaborated on a paper (slightly) within the 4-year window on this early 2020 publication (submitted Oct 2019): https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019GL085792

I note that the level of our interaction was minimal (we were advising on use of various component models, mediated by the lead authors at University of Illinois), and the actual research work was completed outside of the 4-year COI window. When accepting the review assignment, it hadn't occurred to me that Dr. Hartin and I might have a COI since it's been so long.

Thought it best to bring this up in the public review forum given the potentially sensitive nature of this work. I'm happy to proceed however you feel is best.

pdebuyl commented 1 year ago

Hello @tonyewong thanks a lot for mentioning this, I had indeed not noticed it.

I will check with other editors about this and let you know if we may proceed as is or not.

pdebuyl commented 1 year ago

Hello @tonyewong the review can proceed without issue. Given the weak link and almost "expired" age of the link, there is no COI issue.

pdebuyl commented 1 year ago

@hahsan1 , gentle reminder about the review in progress.

hahsan1 commented 1 year ago

Review checklist for @hahsan1

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/USEPA/FrEDI/?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@knoiva-indecon) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

hahsan1 commented 1 year ago

I have successfully installed the package and tested out the first example available in the link provided (usepa.github.io/FrEDI). I tried to use a dummy file I made for tempInputFile, but I'm getting an error after running the import_inputs() function:

In import_inputs():
    User supplied temperature input...
        Importing data from c(2000, 2001, 2002, 2003, 2004, 2005)...        Importing data from c(25, 26.4, 24, 23.6, 33, 23)...
Error in file.exists(.) : invalid 'file' argument

Is the format incorrect?

Also in Step 3, the write output commands should use output_df$results since it's a list.

In terms of the manuscript, can you speak to any other packages or code that is used in this application and how it compares to FrEDI?

tonyewong commented 1 year ago

Regarding the license file -- I see that it's present in the joss branch, but not in main. Should that be included in the main branch too? Since presumably that will be the one that users would access and need to know the license info.

Edit: just noticed it's prominent in the documentation site, just hadn't gotten there yet. @pdebuyl can advise if this is a requirement for the github repo, or if the license in the documentation site is sufficient.

tonyewong commented 1 year ago

The some of the vignettes from the GitHub Pages documentation site, the kableExtra package is loaded and used. I may have missed if elsewhere this dependency is noted and the pacakge is installed, and this isn't a core part of the FrEDI package itself. But, could be nice to include in the initial package installations so it doesn't trip folks up later potentially.

I also noticed that the top-level README.md in the Github repo is very bare bones. This is probably fine since the documentation site is very thorough. The "about" in the upper-right of the github repo links to it, but could we add a statement in the README.md that explicitly tells newcomers to check out the documentation site for further info and examples?

tonyewong commented 1 year ago

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

For community guidelines, it looks like the contributions part of the documentation is still under construction. Is this ready to be added?

pdebuyl commented 1 year ago

Hi all, thanks for the progress in the review!

Regarding the license it should be in the repository. BTW, it is indeed nice to have a self-sufficient (even small) README file, pointing to the docs for more info.

pdebuyl commented 10 months ago

@knoiva-indecon there was quite some work here in the review of your contribution. Can you take the feedback into account and provide an update here?

kthyng commented 9 months ago

@knoiva-indecon Can you chime in here to give an update?

pdebuyl commented 9 months ago

Hi @kthyng @tonyewong @hahsan1 I received news from @knoiva-indecon that her department at EPA is busy with a reporting period and that she will come back to this submission in the coming weeks.

kthyng commented 8 months ago

@pdebuyl Ok I will add the "paused" label to reflect your note.

crvernon commented 4 months ago

:wave: @pdebuyl - are we ready to move forward with this review? Usually we only pause for a month or so. It looks like your reviewers had made good progress.

pdebuyl commented 4 months ago

Hello @crvernon the author will provide replies by the end of next week.

kthyng commented 1 month ago

It has been several more months since the previous check in. Can we have a status update?

pdebuyl commented 4 weeks ago

Hi @kthyng the package underwent important review within the administration recently from what I read at https://usepa.github.io/FrEDI/index.html . I emailed the author today and hope that they will be able to reply to us soon now.

pdebuyl commented 1 week ago

Hello @kthyng I had no reply, please close the review.

@tonyewong , @hahsan1 thank you for your work in this review, sorry that it did not complete.

kthyng commented 1 week ago

@pdebuyl Ok thank you. I will withdraw this submission.

kthyng commented 1 week ago

@editorialbot withdraw

editorialbot commented 1 week ago

Paper withdrawn.

openjournals / joss-reviews