[REVIEW]: PIVC: A C/C++ Program for Particle Image Velocimetry Vector Computation

whedon commented 2 years ago

Submitting author: !--author-handle-->@fibreglass2@Kevin-Mattheus-Moerman<!--end-editor-- Reviewers: @clarka34, @quynhneo Archive: 10.5281/zenodo.7556040

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/9e69f9bdc717641df7cb41c3590f0ae4"><img src="https://joss.theoj.org/papers/9e69f9bdc717641df7cb41c3590f0ae4/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/9e69f9bdc717641df7cb41c3590f0ae4/status.svg)](https://joss.theoj.org/papers/9e69f9bdc717641df7cb41c3590f0ae4)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@timdewhirst & @clarka34 & @quynhneo, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @Kevin-Mattheus-Moerman know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @timdewhirst

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@fibreglass2) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @clarka34

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@fibreglass2) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @quynhneo

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@fibreglass2) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

quynhneo commented 2 years ago

I added a couple of comments on gitlab

OSX build: seems like an outstanding issue https://gitlab.com/fibreglass/pivc/-/issues/22
documentation: Unix dependencies list and instruction https://gitlab.com/fibreglass/pivc/-/issues/11

Kevin-Mattheus-Moerman commented 2 years ago

I added a couple of comments on gitlab

1. OSX build: seems like an outstanding issue

2. documentation: Unix dependencies list and instruction

@quynhneo thanks. Can you link to the issues on GitLab here too please?

fibreglass2 commented 2 years ago

@fibreglass2 Can you comment

Style/Structure: The first point the reviewer raised on the gitlab tracker (https://gitlab.com/fibreglass/pivc/-/issues/3) discusses this. PIVC is designed as standalone software to perform one task, and not a generalized building block style system that is intended to be expanded over time or perform functions outside of its original design goal.

C++ Usage: This software was never intended to be written primarily or entirely in C++. However, work to use more C++ where suitable has been performed at the reviewers request. This includes file type support, improvements in efficiency, and improvements in RAM. See https://gitlab.com/fibreglass/pivc/-/issues/3 https://gitlab.com/fibreglass/pivc/-/issues/15 https://gitlab.com/fibreglass/pivc/-/issues/14 https://gitlab.com/fibreglass/pivc/-/issues/5 https://gitlab.com/fibreglass/pivc/-/issues/4

Testing: detailed tests were added in a previous update. See https://gitlab.com/fibreglass/pivc/-/issues/3 https://gitlab.com/fibreglass/pivc/-/issues/6

OS X and GitLab CI/CD: Presently there is no explicit support for OS X operating systems. The authors have neither familiarity with nor access to OS X for testing and development. It is certainly possible to produce a pre-compiled OS X version (cross compiling currently under investigation) and this is an area under development. Similarly, using the CI/CD system on GitLab is currently under development.

Documentation: This is the present main area of focus.

Kevin-Mattheus-Moerman commented 2 years ago

@clarka34, @quynhneo. :wave: could you give me an update on review progress? Also let us know if you are waiting for @fibreglass2 to make changes. Thanks again for your help. @fibreglass2 let us know if there are any updates from your end.

fibreglass2 commented 2 years ago

@clarka34, @quynhneo. wave could you give me an update on review progress? Also let us know if you are waiting for @fibreglass2 to make changes. Thanks again for your help. @fibreglass2 let us know if there are any updates from your end.

Added new documentation to address previously raised issues by the reviewers.

Kevin-Mattheus-Moerman commented 2 years ago

@clarka34, @quynhneo. :wave: could you give me an update on review progress?

Kevin-Mattheus-Moerman commented 2 years ago

@clarka34, @quynhneo. :wave: could you give me an update on review progress?

clarka34 commented 2 years ago

@Kevin-Mattheus-Moerman sorry for the delay! Currently, the documentation seems pretty sparse to me, so I was waiting to see how @fibreglass2 updated the documentation since they mentioned that they are currently working on that. Specifically, I was waiting to see if the example sections get expanded. The example only uses two images, but it would be useful to have some real-world image sets and some explanation of the processed results explaining what users are looking at in the example. Perhaps some of the images for which this code was written?

Kevin-Mattheus-Moerman commented 2 years ago

@fibreglass2 have you worked on the above? :point_up:

Kevin-Mattheus-Moerman commented 2 years ago

@fibreglass2 :wave:

fibreglass2 commented 2 years ago

All,

There is an update to the example files pending. It contains the synthetic images and statistical analysis requested by @quynhneo in https://gitlab.com/fibreglass/pivc/-/issues/18. GitLab limits releases to 10MB hence selection of example images is carefully being considered as each image pair is relatively large. In order to form a meaningful image set, several thousand images are required which is both impractical and not possible to distribute with the software through GitLab. Currently I am working on obtaining images from some previous experiments by other authors where PIVC has been used in order to supplement the existing experimental images in the release.

fibreglass2 commented 2 years ago

A new update has been added to PIVC. This update provides additional experimental images from other authors as requested by the reviewers. Due to size limitations it is not possible to distribute entire experimental image sets with PIVC. In addition, as documented in https://gitlab.com/fibreglass/pivc/-/issues/18 the ability to generate synthetic images (with known solutions) of arbitrary size and quantity. Included is a script to perform a statistical comparison between the analytical solution used to generate the synthetic images and the PIVC computation. I apologize for the delay in this update.

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34, @quynhneo can you please pick up the review process again? It looks like @fibreglass2 made several changes, and responded to your feedback.

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34, @quynhneo :wave: :point_up:

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34 @quynhneo please can you continue your review at your earliest convenience? If you are no longer able to review please let us know.

quynhneo commented 1 year ago

OK there will be some update by OCt 10

clarka34 commented 1 year ago

Sorry for the delay! I no longer work in academia, so I am pretty time constrained. I will try to look at the changes over the weekend.

quynhneo commented 1 year ago

@fibreglass2 Thanks for the effort, especially the MATLAB scripts. I just provided another issue.

clarka34 commented 1 year ago

@fibreglass2 added some additional comments too! I have not looked at the code yet since I have a mac. Planning to borrow a PC to test soon :)

fibreglass2 commented 1 year ago

Thanks to both reviewers for the feedback. I will be going through these in detail over the weekend.

danielskatz commented 1 year ago

👋 @fibreglass2 - Can you update us on this? As the track editor here, I just wanted to check in on on the progress of this review after a couple of weeks with no changes

fibreglass2 commented 1 year ago

wave @fibreglass2 - Can you update us on this? As the track editor here, I just wanted to check in on on the progress of this review after a couple of weeks with no changes

I have addressed all other recent feedback except one regarding a technical issue with a script on windows.

fibreglass2 commented 1 year ago

wave @fibreglass2 - Can you update us on this? As the track editor here, I just wanted to check in on on the progress of this review after a couple of weeks with no changes

I have addressed all other recent feedback except one regarding a technical issue with a script on windows.

This issue has been resolved, hence all recent feedback has been addressed.

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34, @quynhneo looks like changes/updates were made and some of your concerns/issues may be addressed. Can you have another look?

quynhneo commented 1 year ago

I have completed my review and have no further comments. I applaud the authors for continuing efforts. Thanks

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34 can you provide an update too please?

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34 👋

clarka34 commented 1 year ago

Sorry for the delay! I have been swamped with work. I will provide my update by the end of the week.

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34 thanks

clarka34 commented 1 year ago

@Kevin-Mattheus-Moerman I reviewed everything but the actual code (since I am not familiar with C/C++). Everything looks good to me except for a couple of small things summarized in the following issues:

Nice job @fibreglass2

Kevin-Mattheus-Moerman commented 1 year ago

Thanks @clarka34 for your continued help!

@fibreglass2 will you work on these issues? ☝️

fibreglass2 commented 1 year ago

@Kevin-Mattheus-Moerman The newest comments by @clarka34 have been reviewed and addressed.

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34 can you check these changes/updates? ☝️

clarka34 commented 1 year ago

@Kevin-Mattheus-Moerman I looked and sent a follow up message to @fibreglass2. Should be done once he replies and I just recheck that the code runs as promises :)

fibreglass2 commented 1 year ago

@Kevin-Mattheus-Moerman I looked and sent a follow up message to @fibreglass2. Should be done once he replies and I just recheck that the code runs as promises :)

Just responded to your message.

clarka34 commented 1 year ago

Looks good to me! Thanks @fibreglass2 for all your work on this project :)

Kevin-Mattheus-Moerman commented 1 year ago

@clarka34 thanks for your help reviewing this work! So I take it you recommend acceptance at this point?

Kevin-Mattheus-Moerman commented 1 year ago

@quynhneo I see you recommended acceptance. However can you please tick the remaining boxes above :point_up:

fibreglass2 commented 1 year ago

@whedon generate pdf

editorialbot commented 1 year ago

My name is now @editorialbot

fibreglass2 commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fibreglass2 commented 1 year ago

Simply updated a citation in the paper. Thanks to the reviewers for their extensive and detailed feedback.

fibreglass2 commented 1 year ago

@Kevin-Mattheus-Moerman Is this review completed?

Kevin-Mattheus-Moerman commented 1 year ago

@fibreglass2 yes it is. Apologies for the delay! I will now process this for acceptance as soon as possible. There may be come minor edits required from my end of the paper and it should move relatively quickly now.

Kevin-Mattheus-Moerman commented 1 year ago

@fibreglass2 I have read the paper and it looks good to me. A minor note, which you may or may not choose to work on, is that mostly we use open source rather than open-source (if you choose to change it, also change closed-source to closed source in the text). But this is not a requirement.

At this point can you work on the following:

[ ] Can you archive a copy of your software on ZENODO? You can do this manually or follow these steps: https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content
[ ] Can you report the DOI of the ZENODO archive?
[ ] Next check that the metadata for the ZENODO archive matches certain aspects of the paper, in particular the author list (default may be all contributors so edit if needed), the author order, the title (default is the repository name so change it to match the JOSS paper title). You can also add ORCIDs for all researchers on ZENODO there if you like.
[ ] Next inform me if the version tag (currently here set to v1.0.0) is still good or if during review or archiving you'd altered it.

fibreglass2 commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fibreglass2 commented 1 year ago

@Kevin-Mattheus-Moerman

[x] Can you archive a copy of your software on ZENODO?

See archive here

[x] Can you report the DOI of the ZENODO archive?

DOI: 10.5281/zenodo.7556040

[x] Next check that the metadata for the ZENODO archive matches certain aspects of the paper, in particular the author list (default may be all contributors so edit if needed), the author order, the title (default is the repository name so change it to match the JOSS paper title). You can also add ORCIDs for all researchers on ZENODO there if you like.

Author name and title are correct on Zenodo.

[x] Next inform me if the version tag (currently here set to v1.0.0) is still good or if during review or archiving you'd altered it.

Newest version is v1.2-b

The paper has been updated with corrected usage of "open source" and "closed source".

Kevin-Mattheus-Moerman commented 1 year ago

@editorialbot set 10.5281/zenodo.7556040 as archive

openjournals / joss-reviews