[REVIEW]: fastSF: A parallel code for computing the structure functions of turbulence

whedon commented 4 years ago

Submitting author: @ShubhadeepSadhukhan1993 (Shubhadeep Sadhukhan) Repository: https://github.com/ShubhadeepSadhukhan1993/fastSF Version: v1.0.0 Editor: @jedbrown Reviewer: @cpgr, @iljah Archive: 10.5281/zenodo.4420031

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/71ba620016640c5ded6e927af83f4d38"><img src="https://joss.theoj.org/papers/71ba620016640c5ded6e927af83f4d38/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/71ba620016640c5ded6e927af83f4d38/status.svg)](https://joss.theoj.org/papers/71ba620016640c5ded6e927af83f4d38)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@cpgr & @iljah, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @jedbrown know.

✨ Please try and complete your review in the next six weeks ✨

Review checklist for @cpgr

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@ShubhadeepSadhukhan1993) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @iljah

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@ShubhadeepSadhukhan1993) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

jedbrown commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jedbrown commented 4 years ago

The \mathbf l in your paper looks very much like1 in our font. Could you use \bm l (or use \ell) to avoid this ambiguity?
"Phys of Fluids" -> "Physics of Fluids"
Capitalization in the yaml-cpp reference
I don't think the discussion of six vs three nested loops is a great way to describe what's going on. I'd just say you use vectorization. I also don't think this implementation as described is anywhere close to optimal. You're currently reading all the data for each value of l, but you could batch together a tile of those l so that you make many fewer passes through the global data. This has similar structure to GEMM and convolutions, both of which can achieve near-peak floating point. I predict that if you estimate a flops rate for your code (or use a profiler with hardware perf counters) you'll find that it's bottlenecked on memory bandwidth despite the surplus of flops. It's okay if you don't address this, but in that case, you can just say we use this vectorized computation strategy without making specific claims of high performance.

Apart from that, it looks good modulo the software distribution/best-practices points above, after which we'll be ready to archive.

danielskatz commented 3 years ago

👋 @ShubhadeepSadhukhan1993 & @jedbrown - Can I check in on the status of this?

shashwat-bhattacharya commented 3 years ago

Hello @danielskatz , we are working on the suggestions of @jedbrown . We will get back once they are implemented.

shashwat-bhattacharya commented 3 years ago

Hello @jedbrown First, thanks a lot for your suggestions. Please accept our apologies for our delay in getting back to you; we are all working remotely under rather suboptimal conditions because of COVID-19 pandemic.

We agree that asking users to modify Makefile is not a good practice. We wish to clarify that the users will not be required to edit the makefile as long as all the dependencies are installed in any directory that has been added to the user $PATH. We have now clarified this in README.md. Further, we now also mention that the installation instructions in README.md are for installing the dependencies in $HOME/local (which is generally recommended), but the users have the option of installing the dependencies anywhere as long as it is added to the $PATH. Hopefully, this addresses the issue regarding the Makefile.
We fully agree that fastSF should be compatible with more recent versions hdf5. We have successfully executed it with hdf5 v1.10.5 in Shaheen II of KAUST, Saudi Arabia, thereby showing that it works with newer hdf5 versions. Hence, we have removed the disclaimer regarding hdf5 1.8 from README.md.
We have modified the code to read the grid dimensions from the hdf5 files. The grid dimensions are now required to be specified only in the testing mode, where the input files are generated by the code and not read from hdf5 files.
fastSF now responds to -h, in which the options for command-line arguments are displayed to the user.
Although your suggestion is good, there is one potential problem in scanning one or more datafiles looking for datasets with specified names: multiple hdf5 files can contain a dataset with the same name. This indeed is the case with a few CFD solvers that we know of.  Nevertheless, we fully agree that having a constraint of ensuring the same name for the hdf5 file and its dataset is not user-friendly, as the user has to modify the output files of CFD solvers to make them compatible with fastSF. Hence, we remove this constraint, and in the present version of the code, the user can specify the names of both the hdf5 file and the dataset separately during the command run. Thus, the user no longer needs to tinker with output files of CFD solvers (as long as the files are in hdf5 format). Further, the problem of having datasets with the same name in multiple hdf5 files will not arise.

shashwat-bhattacharya commented 3 years ago

Thanks for your suggestions on the manuscript @jedbrown . We have modified it as follows:

We have replaced \mathbf with \boldsymbol everywhere. This should resolve the ambiguity concerning ‘l'.
“Phys of Fluids” has been corrected to “Physics of Fluids”. Thanks for pointing this out and sorry for this typo!
We have removed the capitalization in the yaml-cpp reference.
Thanks a lot for your suggestion in improving the performance of the code. However, currently, it will be difficult to address this as we are all working remotely. However, we will keep upgrading the code and will definitely incorporate your suggestion. We have removed specific claims of high performance its well as the discussion of six vs. three nested loops in the revised manuscript.

Thank you again for your and the referees’ constructive feedback. We feel there has been a significant improvement in the code compared to the version that was first submitted to JOSS thanks to your valuable inputs. We hope that our work is now suitable for publication.

jedbrown commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jedbrown commented 3 years ago

Thanks for your patience. This is great.

"Structure function is a popular diagnostics tool to study the statistical properties..." -- this grammar is awkward. Perhaps "Structure functions are popular diagnostics in the study of statistical properties ..."
line 22 : "fastSF is a C++ application for ..." is shorter and reads more clearly.
I know "ensemble average" is common terminology, but if I understand correctly, what your code computes is a spatial average and that terminology would be more clear (your code isn't averaging across flow snapshots at different times or with different initial conditions).
I think small figure that plots a structure function (e.g., scalar field in 2D) and the radial representation (which I think your software doesn't compute, but you might want to ship a script just so users can reproduce standard figures without writing code) would be a lot more informative than the implementation details. Your figure 1 is still useful.
The yaml-cpp reference looks mangled.
The h5si reference has a mangled double URL.
high-Reynolds should be high-{Reynolds} to protect capitalization
The hdf5 DOI appears twice

Once you've addressed these points and checked the PDF, it'll be time to archive:

[ ] Make a tagged release of your software, and list the version tag of the archived version here.
[ ] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
[ ] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[ ] Please list the DOI of the archived version here.

shashwat-bhattacharya commented 3 years ago

Thanks @jedbrown for your suggestions. I have implemented all of them except for the one about the figure about the structure functions; I have a small confusion here. Should we include the section on verification in the main paper and shorten the section on the design of the code? Or should we just include a couple of plots for the structure functions of idealized velocity fields with a few lines of explanation in "Velocity and Scalar Structure Functions" section? Thanks a lot for your help.

jedbrown commented 3 years ago

I think plotting a structure function for a real turbulent flow would be most useful. Verification is important for people who are digging deeper, but I think not the thing to promote up-front.

ShubhadeepSadhukhan1993 commented 3 years ago

@jedbrown Wishing you a Happy New Year and thanks for your valuable suggestions. We have added a figure that plots the velocity structure functions of 3D homogeneous isotropic turbulence in section 2 of the paper. We have made a tagged release of our software and then archived it in zenodo as directed by you. The doi of the archived version is 10.5281/zenodo.4420031. Thank you for your careful review of the software.

jedbrown commented 3 years ago

@whedon set 10.5281/zenodo.4420031 as archive

whedon commented 3 years ago

OK. 10.5281/zenodo.4420031 is the archive.

jedbrown commented 3 years ago

@whedon accept

whedon commented 3 years ago

Attempting dry run of processing paper acceptance...

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/acprof:oso/9780198722588.001.0001 is OK
- 10.1017/9781316810019 is OK
- 10.1017/CBO9781139170666 is OK
- 10.1016/C2009-0-18471-4 is OK
- 10.1098/rspa.1991.0076 is OK
- 10.1098/rspa.1991.0075 is OK
- 10.1103/PhysRevLett.70.3251 is OK
- 10.1016/j.jpdc.2017.10.014 is OK
- 10.1063/1.2001690 is OK
- 10.1063/1.1539855 is OK
- 10.1063/1.1448296 is OK
- 10.1088/1367-2630/10/3/033003 is OK
- 10.1088/1367-2630/6/1/037 is OK
- 10.1103/PhysRevE.77.016302 is OK
- 10.1017/jfm.2013.74 is OK
- 10.1146/annurev.fluid.010908.165203 is OK
- 10.1063/1.5119905 is OK
- 10.1103/PhysRevLett.72.336 is OK
- 10.1007/s12043-013-0594-4 is OK
- 10.1103/PhysRevFluids.4.084607 is OK
- 10.1007/3-540-49372-7_24 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 3 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/2023

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/2023, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

jedbrown commented 3 years ago

@cpgr @iljah Thanks for your work reviewing this submission!

@ShubhadeepSadhukhan1993 Thanks for your diligence seeing this through review. I can now recommend it for acceptance, pending final checks from an EiC.

arfon commented 3 years ago

@whedon accept deposit=true

whedon commented 3 years ago

Doing it live! Attempting automated processing of paper acceptance...

whedon commented 3 years ago

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

whedon commented 3 years ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/2026
Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02185
If everything looks good, then close this review issue.
Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

arfon commented 3 years ago

@cpgr, @iljah - many thanks for your reviews here and to @jedbrown for editing this submission. JOSS relies upon the volunteer efforts of folks like yourselves - we simply couldn't do it without you! ✨

@ShubhadeepSadhukhan1993 - your paper is now accepted and published in JOSS :zap::rocket::boom:

whedon commented 3 years ago

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02185/status.svg)](https://doi.org/10.21105/joss.02185)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02185">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02185/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02185/status.svg
   :target: https://doi.org/10.21105/joss.02185

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

openjournals / joss-reviews