openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[REVIEW]: SPICY: A python toolbox for meshless assimilation from image velocimetry using radial basis functions #5749

Closed editorialbot closed 9 months ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@mendezVKI<!--end-author-handle-- (Miguel Alfonso Mendez) Repository: https://github.com/mendezVKI/SPICY_VKI Branch with paper.md (empty if default branch): Version: 1.0.5 Editor: !--editor-->@philipcardiff<!--end-editor-- Reviewers: @nolankucd, @MatthewFlamm, @ctdegroot Archive: 10.5281/zenodo.10473329

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/ca453152e765cab3bab8a8b57f6aa400"><img src="https://joss.theoj.org/papers/ca453152e765cab3bab8a8b57f6aa400/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/ca453152e765cab3bab8a8b57f6aa400/status.svg)](https://joss.theoj.org/papers/ca453152e765cab3bab8a8b57f6aa400)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@nolankucd & @MatthewFlamm & @ctdegroot, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @philipcardiff know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Checklists

📝 Checklist for @nolankucd

📝 Checklist for @MatthewFlamm

📝 Checklist for @ctdegroot

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago
Software report:

github.com/AlDanial/cloc v 1.88  T=1.42 s (55.6 files/s, 96208.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                      24          10477          24993          76426
HTML                            15            911             38           6885
Python                           5            811           2072           3048
SVG                              1              0              0           2671
CSS                              9            341            300           2503
Jupyter Notebook                 8              0           3411           1127
YAML                             2             26             43            205
reStructuredText                10             59             34             98
TeX                              1              8              0             81
Markdown                         2             35              0             59
DOS Batch                        1              8              1             26
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            79          12680          30899          93138
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 748

editorialbot commented 1 year ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1088/1361-6501/ac70a9 is OK
- 10.1017/s0962492914000130 is OK
- 10.1007/s00348-016-2133-9 is OK
- 10.1007/s00348-021-03172-0 is OK

MISSING DOIs

- 10.18409/ispiv.v1i1.198 may be a valid DOI for title: Main results of the first Data Assimilation Challenge
- 10.55037/lxlaser.20th.115 may be a valid DOI for title: A RANS approach to the Meshless Computation of Pressure Fields From Image Velocimetry

INVALID DOIs

- https://doi.org/10.1016/j.taml.2020.01.039 is INVALID because of 'https://doi.org/' prefix
editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

philipcardiff commented 1 year ago

Hi @mendezVKI, please address the missing and invalid DOIs above; thanks!

philipcardiff commented 1 year ago

Also, please look at the pull request I created (https://github.com/philipcardiff/SPICY_VKI/pull/1, fix: https://github.com/mendezVKI/SPICY_VKI/pull/7), which aims to fix an issue with the first reference.

MatthewFlamm commented 1 year ago

Review checklist for @MatthewFlamm

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

philipcardiff commented 1 year ago

Hi @nolankucd and @ctdegroot, a reminder that you can generate your reviewer checklist by running this command in a separate comment:

@editorialbot generate my checklist
MatthewFlamm commented 1 year ago

Installation requires: https://github.com/mendezVKI/SPICY_VKI/pull/4

The dependencies of the package are also a little messy, and this results in unneeded packages being installed, ~not as critical~: https://github.com/mendezVKI/SPICY_VKI/issues/3

Edit: I now think both are important as some dependencies might be required to run tutorials for verifying functionality.

MatthewFlamm commented 1 year ago

I could not find any community guidance in the repo/documentation. Tracked in https://github.com/mendezVKI/SPICY_VKI/issues/8

MatthewFlamm commented 1 year ago

I have gone through the checklist and consider my review complete, pending needed changes to the paper and repository. High level is that I believe this submission solves a useful scientific need. The tutorials are of generally good quality, although I was not able to run the 3D case on my machine due to memory constraint. In the future, I would suggest to the authors to think about how the package could become easier to use for users as the workflow seems to have many common elements in each case. Today, the code does provide useful tools for researchers in this area. However, the repository is currently messily organized (see https://github.com/mendezVKI/SPICY_VKI/issues/2) and missing several key items for acceptance today. See below on specifics.

Other than what I have already commented above, these are my comments relating to the last items: Major comments:

Minor comments:

mendezVKI commented 1 year ago

Thank you so much for your feedback and wonderful work, Matthew.

I will address all points early next week.

All the best,

philipcardiff commented 1 year ago

I have gone through the checklist and consider my review complete, pending needed changes to the paper and repository. ...

Thanks for your comprehensive review, @MatthewFlamm!

mendezVKI commented 1 year ago

Dears, thank you so much for your detailed review.

I have addressed all points in the issues. Concerning the ones open in this check list:

  1. References: I modified a bit them, removing the one that was creating problem and added others concerning open-software for PIV/PTV. There are still a couple of references that do not have a DOI; this is because these are conf papers. I considered for a moment removing these, but I feel that there is very useful information there so I prefer to keep them.

  2. A statement of need: I add a sentence mentioning open-source software and about positioning SPICY in the larger landscape. Although some authors (e.g. the cited Rao 2020) shared a github ref with their implementation of the assimilation using PINNs, we have not found any software package similar to SPICY. We state this clearly. Concerning the lack of "a clear description of who the software is intended to be used by", we added a clear statement in the readme but can't fit it in the 2 page format of the article without removing important references. The section reports the main context (data assimilation for velocimetry) that the SPICY package targets, so it is implicit that it targets anyone interested in this topic.

  3. Automated tests: We will consider this for the next release; thank you very much for the suggestion. As beginners in software developments we limited ourselves to tutorials designed to test all the key functions. In our experience, it seems that users learn how to use the code quite quickly while having a good grasp of the underlying numerics from the videos we provided on youtube (https://www.youtube.com/@spicyVKI).

  4. Community guidelines. Indeed, these were missing. We added an .md file with instructions (as per solved issue)

  5. Installation instructions and related documentation. Indeed, several dependencies were unclear. We fixed this as suggested in the issue. We now include an installation with optional dependencies for the tutorials.

The folder should now be clean. I twine upload everything in a few hours.

Once again, thank you so much for the very detailed review and suggestions :)

philipcardiff commented 1 year ago

Hi, again, @nolankucd and @ctdegroot, another reminder about this review. Let us know if you have a timeline in mind. Thanks.

philipcardiff commented 1 year ago

Hi @nolankucd and @ctdegroot, another reminder about this review. Thanks.

philipcardiff commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1088/1361-6501/ac70a9 is OK
- 10.1016/j.cageo.2019.03.007 is OK
- 10.5334/jors.101 is OK
- 10.6084/M9.FIGSHARE.12330608 is OK
- 10.5334/jors.334 is OK
- 10.1017/s0962492914000130 is OK
- 10.1007/s00348-016-2133-9 is OK
- 10.1007/s00348-021-03172-0 is OK

MISSING DOIs

- 10.18409/ispiv.v1i1.198 may be a valid DOI for title: Main results of the first Data Assimilation Challenge
- 10.55037/lxlaser.20th.115 may be a valid DOI for title: A RANS approach to the Meshless Computation of Pressure Fields From Image Velocimetry

INVALID DOIs

- https://doi.org/10.1016/j.taml.2020.01.039 is INVALID because of 'https://doi.org/' prefix
philipcardiff commented 1 year ago

Hi @nolankucd and @ctdegroot, no doubt you are both very busy; if possible, can you indicate an expected timeline for the start of your review? Thanks!

@MatthewFlamm, as your review is complete, please tick all remaining unticked boxes in your checklist, assuming you agree they are complete.

MatthewFlamm commented 1 year ago

I was waiting for other reviews based on the previous comments. I will re check based on the recent changes.

philipcardiff commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

  • 10.1088/1361-6501/ac70a9 is OK
  • 10.1016/j.cageo.2019.03.007 is OK
  • 10.5334/jors.101 is OK
  • 10.6084/M9.FIGSHARE.12330608 is OK
  • 10.5334/jors.334 is OK
  • 10.1017/s0962492914000130 is OK
  • 10.1007/s00348-016-2133-9 is OK
  • 10.1007/s00348-021-03172-0 is OK

MISSING DOIs

  • 10.18409/ispiv.v1i1.198 may be a valid DOI for title: Main results of the first Data Assimilation Challenge
  • 10.55037/lxlaser.20th.115 may be a valid DOI for title: A RANS approach to the Meshless Computation of Pressure Fields From Image Velocimetry

INVALID DOIs

Hi @mendezVKI, please fix these missing and invalid references. Thanks!

mendezVKI commented 12 months ago

I apologize for the late response; I missed this update.

The reference "Main results of the first Data Assimilation Challenge" has been removed. Did something go wrong with the resubmission ? I attach the latest pdf I find in my "Action-> Artifacts" session paper.pdf

The references that do not have a DOI are conference papers for which this info is not relevant. Nevertheless, these are still very valuable to this work and we wish to cite these (papers are available on arxiv).

Edit: I corrected the 10.1016/j.taml.2020.01.039 reference.

philipcardiff commented 12 months ago

Thanks,

@editorialbot check references

philipcardiff commented 12 months ago

@editorialbot check references

editorialbot commented 12 months ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1088/1361-6501/ac70a9 is OK
- 10.1016/j.cageo.2019.03.007 is OK
- 10.5334/jors.101 is OK
- 10.6084/M9.FIGSHARE.12330608 is OK
- 10.5334/jors.334 is OK
- 10.1017/s0962492914000130 is OK
- 10.1016/j.taml.2020.01.039 is OK
- 10.1007/s00348-016-2133-9 is OK
- 10.1007/s00348-021-03172-0 is OK

MISSING DOIs

- 10.18409/ispiv.v1i1.198 may be a valid DOI for title: Main results of the first Data Assimilation Challenge
- 10.55037/lxlaser.20th.115 may be a valid DOI for title: A RANS approach to the Meshless Computation of Pressure Fields From Image Velocimetry

INVALID DOIs

- None
philipcardiff commented 12 months ago

@editorialbot generate pdf

editorialbot commented 12 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

MatthewFlamm commented 12 months ago

I got busy in the last few weeks, but was able to get back to this. I have checked all the boxes.

While there is no explicit statement of who the software is intended to be used by, I think the article and tutorials provide enough context, especially given the expanded discussion of other software packages.

I have added an issue https://github.com/mendezVKI/SPICY_VKI/issues/9, which highlights a typo for installation instructions. But given the substantial changes to the readme and clear installation instructions there, this doesn't hold up my approval on this point.

mendezVKI commented 11 months ago

Hello,

I believe all points have been addressed. May I kindly ask what is the status of the submission and what is the expected next step ?

Yours,

ctdegroot commented 11 months ago

Review checklist for @ctdegroot

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

ctdegroot commented 11 months ago

I have completed my review as well. Apologies, I forgot to come back and fill the checklist.

philipcardiff commented 11 months ago

I have completed my review as well. Apologies, I forgot to come back and fill the checklist.

Thanks @ctdegroot!

mendezVKI commented 10 months ago

Hello @philipcardiff ,

I hope you are having a wonderful Christmas break.

Do you have an estimation of the timing before final publication? I am asking because we would like to cite this article in several upcoming works that used the software.

All the best

Miguel

nolankucd commented 10 months ago

Review checklist for @nolankucd

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

nolankucd commented 10 months ago

Review checklist for @nolankucd

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

nolankucd commented 10 months ago

Apologies for the late review.

philipcardiff commented 10 months ago

Apologies for the late review.

Thanks @nolankucd, is your review complete? Also, you can tick the "Human and animal research" box, as I believe it does not apply to this submission.

philipcardiff commented 10 months ago

Hello @philipcardiff ,

I hope you are having a wonderful Christmas break.

Do you have an estimation of the timing before final publication? I am asking because we would like to cite this article in several upcoming works that used the software.

All the best

Miguel

Hi Miguel,

Apologies for the delay. Once @nolankucd confirms his review is complete, we can move to the final publication stages, which should be relatively quick.

Philip

[Edit: spelling correction]

nolankucd commented 10 months ago

Hi @philipcardiff, yes my review is complete.

philipcardiff commented 10 months ago

@editorialbot check references

philipcardiff commented 10 months ago

@editorialbot generate pdf

editorialbot commented 10 months ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1088/1361-6501/ac70a9 is OK
- 10.1016/j.cageo.2019.03.007 is OK
- 10.5334/jors.101 is OK
- 10.6084/M9.FIGSHARE.12330608 is OK
- 10.5334/jors.334 is OK
- 10.1017/s0962492914000130 is OK
- 10.1016/j.taml.2020.01.039 is OK
- 10.1007/s00348-016-2133-9 is OK
- 10.1007/s00348-021-03172-0 is OK

MISSING DOIs

- 10.18409/ispiv.v1i1.198 may be a valid DOI for title: Main results of the first Data Assimilation Challenge
- 10.55037/lxlaser.20th.115 may be a valid DOI for title: A RANS approach to the Meshless Computation of Pressure Fields From Image Velocimetry

INVALID DOIs

- None
editorialbot commented 10 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

philipcardiff commented 10 months ago

Hi @mendezVKI, please check the article again for any final changes. Once done, please issue a new tagged release of the software (if changed) and archive it (e.g. on Zenodo, figshare, or others). Please then post the version number and archive DOI here.

mendezVKI commented 10 months ago

Hello Philippe,

The article looks good. We just uploaded a minor update to correct the display of the German "ö" in a bib entry. The current release in PyPi is the final one; we just tested and works. The folder was uploaded on Zenodo here:

https://zenodo.org/records/10473329

philipcardiff commented 10 months ago

@editorialbot set 10.5281/zenodo.10473329 as archive

editorialbot commented 10 months ago

Done! archive is now 10.5281/zenodo.10473329

philipcardiff commented 10 months ago

Hello Philippe,

The article looks good. We just uploaded a minor update to correct the display of the German "ö" in a bib entry. The current release in PyPi is the final one; we just tested and works. The folder was uploaded on Zenodo here:

https://zenodo.org/records/10473329

Thanks @mendezVKI. Could you add a tag to the main SPICY_VKI GitHub repository? i.e. add a "1.0.5" tag for the latest code. In this way, users can be sure they are using the exact repository in this article. Thanks.

philipcardiff commented 10 months ago

@editorialbot set 1.0.5 as version