[REVIEW]: RESOURCECODE: A Python package for statistical analysis of sea-state hindcast data

editorialbot commented 2 years ago

Submitting author: !--author-handle-->@NRaillard@marcosvital<!--end-editor-- Reviewers: @krober10nd, @platipodium, @malmans2 Archive: 10.5281/zenodo.7681494

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e43574f4a0b6782ee6a112180912dae0"><img src="https://joss.theoj.org/papers/e43574f4a0b6782ee6a112180912dae0/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e43574f4a0b6782ee6a112180912dae0/status.svg)](https://joss.theoj.org/papers/e43574f4a0b6782ee6a112180912dae0)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@krober10nd & @platipodium & @malmans2, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @marcosvital know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @platipodium

📝 Checklist for @malmans2

📝 Checklist for @krober10nd

editorialbot commented 2 years ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 2 years ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.16 s (403.7 files/s, 44325.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          37            788           1585           2024
JSON                             4              0              0           1015
XML                              1              0              2            441
Markdown                         3            121              0            338
YAML                             2             18              0            142
reStructuredText                11             84             67            131
INI                              3             10              0             62
TeX                              1              7              1             62
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            63           1032           1662           4224
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/j.oceaneng.2019.05.016 is OK
- 10.1016/j.oceaneng.2013.04.015 is OK
- 10.25080/Majora-92bf1922-00a is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 2 years ago

Wordcount for paper.md is 1099

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

marcosvital commented 2 years ago

Dear @krober10nd, @platipodium and @malmans2, thank you again for accepting review this submission for JOSS. The reviewing process is checklist based, and instructions were already posted above by the editorial bot - but let me know if you need any assistance, ok? Also, you can tag @NRaillard if you have specific questions about the manuscript.

@NRaillard, you can tag any of your co-authors if you want, so they would be able to follow this issue.

platipodium commented 2 years ago

Review checklist for @platipodium

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://gitlab.ifremer.fr/resourcecode/resourcecode?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@NRaillard) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

platipodium commented 2 years ago

@NRaillard please provide instructions on how / login to to create issues in your upstream repo.

platipodium commented 2 years ago

The repository's author report is as follows:

Author	Commits (%)	+ lines	- lines	First commit	Last commit	Age	Active days	# by commits
Simon Chabot	138 (65.09%)	307176	249010	2020-07-23	2021-11-23	487 days, 23:04:26	46	1
RAILLARD	41 (19.34%)	1915	600	2021-11-24	2022-03-25	121 days, 1:12:41	5	2
Noe Gaumont	16 (7.55%)	44869	466	2021-08-26	2021-09-16	21 days, 1:31:28	5	3
ogiorgis	12 (5.66%)	2884	1277	2021-07-07	2021-07-13	5 days, 21:26:16	4	4
Nicolas RAILLARD	3 (1.42%)	3	3	2022-01-25	2022-01-25	0:16:07	1	5

The authors on the paper, however, are Nicolas Raillard1¶ , Christophe Maisondieu1 , David Darbynian2 , Gregory Payne∗ 3 , and Louis Papillon

There is some inconsistency in authorship and attributed that blocks acceptance and need to be better explained/corrected.

The submitting author @nraillard contributed 20% to the software, but is the sole author named in citation.cff
Simon Chabot as the main contributor to source is not acknowledged anywhere, neither is Noe Guamont or ogiorgis
The contribution role of Maisondieu, Darwinian, Panye and Papillon is not clear

NRaillard commented 2 years ago

Hello @platipodium thanks for taking some time for the review.

To login on our self-hosted gitlab instance, I need to create an account: I just need your name and email address to do so. I can do also for @krober10nd and @malmans2 as soon as they send me the requested information (by mail or DM)

To clarify the authorship, some of the modules were converted from matlab, Fortran and R codes developed by the authors (C. Maisondieu, D. Darbynian, G. Payne, L. Papillon and myself) as part of a subcontracting to develop the python package. Most of the authors do not use git so do not appear in the aforementioned report but made the most significant contribution to the project.

However, looking at the table, it is clear that Simon Chabot did the most part of the job and should be credited for that, even if the copyright is transfered to IFREMER. I will see with him if he want to be credited and modify the paper accordingly.

platipodium commented 2 years ago

Thank you @NRaillard for starting to answer the authorship issue. Let's continue this in your repository once we all register there. We will have to talk about the separate issues of (1) Copyright and (2) Intellectual property. Also note that in the current code there remain about 50 k lines form Simon and 50 k lines from Noe, and about 2 k from yourself and ogiorgis.

malmans2 commented 2 years ago

Review checklist for @malmans2

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://gitlab.ifremer.fr/resourcecode/resourcecode?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@NRaillard) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

platipodium commented 2 years ago

[x] Issue https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/8 on copyright

platipodium commented 2 years ago

[x] Issue https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/9 on authorship

platipodium commented 2 years ago

[x] Issue https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/10 on missing community guidelines

NRaillard commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

NRaillard commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

platipodium commented 2 years ago

[ ] Issue https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/10 on missing community guidelines

Contribution copyright retention / transfer not yet resolved. Please make clear how you would like to accept contributions. Read all about pros and cons at http://harmonyagreements.org/index.html

marcosvital commented 2 years ago

Dear @krober10nd, I noticed that you still don't have a reviewing checklist yet, so I'm not sure if you already had the chance to start revising this submission. Let me know if you have any problems and if you believe that you can start that soon, ok?

krober10nd commented 2 years ago

Review checklist for @krober10nd

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://gitlab.ifremer.fr/resourcecode/resourcecode?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@NRaillard) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

krober10nd commented 2 years ago

@marcosvital I'll get started in the next week. Sorry for the delay.

marcosvital commented 2 years ago

@marcosvital I'll get started in the next week. Sorry for the delay.

No problem, and thank you for letting us know.

platipodium commented 2 years ago

[x] Installation fails on aarch64-conda-py3.10 https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/13

platipodium commented 2 years ago

[x] Installation fails on aarch64-conda-py3.9 https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/14

platipodium commented 2 years ago

One item remains as a todo before I can finish my review, see https://gitlab.ifremer.fr/resourcecode/resourcecode/-/issues/16

[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?

platipodium commented 2 years ago

I have finished my review and recommend to publish the paper @marcosvital

NRaillard commented 2 years ago

Many thanks @platipodium for the very valuable comments !

NRaillard commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

NRaillard commented 2 years ago

@malmans2 I managed to solve issues raised, did you manage to test further the toolbox ?

We are planning to use the toolbox for other work so we would like to finish this paper, , I will appreciate it if the reviews are concluded soon.

marcosvital commented 2 years ago

Thank you for you effort reviewing this submission, @platipodium.

marcosvital commented 2 years ago

Hi, @krober10nd and @malmans2! Let us know if you can still continue reviewing this submission, ok? And please let me know if you need any help.

malmans2 commented 2 years ago

Hi @marcosvital, apologies for the late reply!

@NRaillard thanks for addressing my previous comments. I have a couple of additional comments/issues, then I think the package is ready for publication:

[x] I'm not able to check whether the CI is properly working. I can see the CI configuration files in the repository, but I can't find a link to the actual CI actions/logs. Could you please share the link?
[x] I've opened 2 new issues: #17, #18
[x] Please merge the dev branches and close the following issues: #11, #12

NRaillard commented 2 years ago

thanks @malmans2 for the updates.

I have given you the 'reporter' role, so that you can now access the CI/CD actions and logs here.
I have closed #11 and #12 and will merge the 'joss_branch' if the package is accepted;
I corrected #17 and responded to #18, let me know if it is OK with you.

malmans2 commented 2 years ago

@NRaillard could you please review MR #2? It should address the points I've raised in Issue #17

NRaillard commented 2 years ago

Works like a charm, thanks a lot: I wasn't able to make `pytest --doctest-glob README.md' run on my own...

malmans2 commented 2 years ago

@marcosvital I also finished my review. The package is in great shape and I think it's ready for publication!

marcosvital commented 2 years ago

Thank you so much for your effort, @malmans2.

marcosvital commented 2 years ago

@krober10nd, please let us know if you are still able to review this submission, ok? And let me know if you need any assistance.

krober10nd commented 2 years ago

Hello, Sorry I apologize for the delay. I am currently unable to install the package on Python 3.10 following the instructions on the source codes' webpage. I also cannot post issues without registering on gitlab. Are there specific versions of Python this software is compatible with? If so, it would be helpful to list them on the landing page.

What are the advantages of gitlab over GitHub? Seems like for user adoption a natural deterrent and naturally would recommend this code be migrated over to a platform that the majority of users frequent and use.

My other major concern for this package is how it can be generalized to other wave hindcasts or hydrodynamic water level hindcasts. In its current state it appears specifically geared towards the one dataset the authors' developed and validated, however performing wave and water water level hindcasts are common engineering tasks. Is it possible to extend this tool to other datasets? How would one go doing this and is there a flexible API in place for dealing with this? Also, can this be expanded on in the paper please?

NRaillard commented 2 years ago

Dear Keith,

Python 3.10 is not supported yet as pyarrow binaries are not available. The dependencies are listed on the JOSS branch (https://gitlab.ifremer.fr/resourcecode/resourcecode/-/tree/joss_paper) and the landing page will be modified when this branch is merged with the main, if the paper is accepted.

IMHO, there are no real advantages of gitlab over github, except that our organisation offers a self-hosted gitlab that we are encouraged to use. There are currently no plans to move the code outside this gitlab instance. One drawback, is that we must ask for the creation of an account for you so that you can create issues, clone, merge, review the CICD... you can send me an email and I will ask for its creation.

As the package is already available on pypi, i don't think that having the code on gitlab instead of github can block the adoption of the package.

Concerning the last point, one part of the package is indeed devoted to the RESOURCECODE wave hindcast, however the major part of the code (environmental contours, weather windows, production assessment...) only assumes a time series of sea-state parameters or wave spectrum and is general enough to deal with any data source, as long as the data is loaded properly in a pandas dataframe. It has been developed and tested on the precursor of RESOURCECODE, the HOMERE hindcast data, which was covering only the french coast and on in-situ measurements data from the test sites. I can add a precision in the paper.

NRaillard commented 2 years ago

Dear @krober10nd, just a kind reminder that if you want to have access to the CI/CD configuration, post issues for questions/remarks about the code, I just need you email so that I can ask for the creation of a personal account on our organization gitlab instance.

krober10nd commented 2 years ago

@NRaillard okay my email is kroberts@baird.com thanks

NRaillard commented 2 years ago

Ok, my bad: it seems that you already have an account on our gitlab with another email ( @usp). You should have received an emil on this email with a password to access gitlab.ifremer.fr Let me know if you have some issues in connecting.

krober10nd commented 2 years ago

I no longer have access to my institution account unfortunately.

krober10nd commented 2 years ago

I've posted several issues on the gitlab website. I was not able to install on Python 3.10 (as you said),Python 3.9, nor Python 3.8. I had to use Python 3.7 to install.

In its current state, I cannot recommend acceptance of this article but I hope this changes with revision and further effort on the software side.

Unless I'm missing something, the software appears to gather data from a server and load it into a pandas dataframe. There's some modules for extreme value modeling but it largely remains unclear how these are used. More examples of its usage especially in regard to the advertised features in the paper as I mentioned in my issue, are necessary.

As @NRaillard alluded to above the code could be used for other datasets but again, without simple and short examples regarding how to do these analyses (e.g., (environmental contours, weather windows, production assessment...), it's unfortunately not very helpful.

NRaillard commented 2 years ago

Thank you Keith for your feeback. I have answered the issues on the gitlab.

I'm a bit surprised that you were not able to install the package on python 3.9, since I am using it. Could you provide me some details about the issues ? Indeed the package offers te possibility to download data from a server, but offers some more functions (metadata such as nodes locations, variables available...), helpers to find locations, conversion from 2D spectral data to sea-state parameters, environmental contours, resource assessment... The documentation may be rather crude at some point, but as stated in the paper, we also developed a web portal with some use-cases of the toolbox: https://resourcecode.ifremer.fr/tools

NRaillard commented 2 years ago

A quick update about the tested versions: I managed to install the toolbox (with pip) on python 3.8, & 3.7 in addition to 3.9. It fails on python 3.6 because of missing dependencies.

openjournals / joss-reviews