[REVIEW]: GeophysicalFlows.jl: Solvers for geophysical fluid dynamics problems in periodic domains on CPUs & GPUs

whedon commented 3 years ago

Submitting author: @navidcy (Navid C. Constantinou) Repository: https://github.com/FourierFlows/GeophysicalFlows.jl Version: v0.12.1 Editor: @pdebuyl Reviewer: @ranocha, @eviatarbach Archive: 10.5281/zenodo.4695260

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/a8cdf26beae8bcecc751ab4ded53b308"><img src="https://joss.theoj.org/papers/a8cdf26beae8bcecc751ab4ded53b308/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/a8cdf26beae8bcecc751ab4ded53b308/status.svg)](https://joss.theoj.org/papers/a8cdf26beae8bcecc751ab4ded53b308)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@ranocha & @eviatarbach, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @pdebuyl know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @ranocha

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@navidcy) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @eviatarbach

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@navidcy) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

navidcy commented 3 years ago

Thanks @eviatarbach. I’ll fix that about MAOOAM.

navidcy commented 3 years ago

@whedon generate pdf from branch JOSS-paper

whedon commented 3 years ago

Attempting PDF compilation from custom branch JOSS-paper. Reticulating splines etc...

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

pdebuyl commented 3 years ago

Thank you @navidcy for the update.

pdebuyl commented 3 years ago

Thank you @eviatarbach for the re-review! Can you make sure to tick all the remaining boxes for completing the review though? The boxes and explicit acceptance are both part of the review process.

ranocha commented 3 years ago

Thanks to @navidcy and co-workers! This is a nice contribution to the community and everything looks good to me after the revisions :+1:

eviatarbach commented 3 years ago

@pdebuyl Done! Thank you.

pdebuyl commented 3 years ago

Thank you @ranocha and @eviatarbach for the review!

@navidcy I will now take over for the next step of the editorial procedure https://joss.readthedocs.io/en/latest/editing.html#after-reviewers-recommend-acceptance . When I am done (after possibly asking minor changes), I'll notify the eic for publication.

glwagner commented 3 years ago

Could you please explain your reasons for creating several submodules such as TwoDNavierStokes? A more Julian approach seems to be to create some equations type and pass that around using multiple dispatch. That would probably make it easier to use GeophysicalFlows.jl as a library with multiple physical setups.

Indeed, some of the modules might have overlapping functionality. For example, the equations that TwoDNavierStokes module solves is reproduced by SingleLayerQG when setting β=0, \ell = 0 and no topography. The reason we have done so was party because different disciplines feel more comfortable, i.e., mathematicians working with the 2D Navier-Stokes or 2D Euler equations don't feel quite "at home" when presented with the "single layer quasi-geostrophic equations". We see the reviewer's point thought that we could have some common functionality under the hood and then the various modules use multiple dispatch to use and/or enhance that functionality. This, though, requires a big restructure of the package and we decided to postpone it for future. Happy to discuss more on this.

I apologize for the late response --- I just wanted to add to @navidcy's response!

I think it's a valid criticism that the design of GeophysicalFlows.jl / FourierFlows.jl isn't built around multiple dispatch but instead relies on passing functions around (in particular, FourierFlows.Equation stores the function calcN! as data). I'd argue, however, that whether or not the code is organized into modules isn't the main issue. Rather than issue is how PDE solvers are implemented in the FourierFlows.jl framework.

To explain the two possibilities we need to distinguish from "module writers" (who are users of FourierFlows.jl, like the writers of GeophysicalFlows.jl modules) and "users" (who use GeophysicalFlows.jl").

The design FourierFlows.jl currently implements is that it defined a fundamental type Problem. Module writers then implement a PDE by building instances of Problem; the crucial function calcN! is stored in Problem.eqn.calcN!. In this design, the PDE being solved can only be inferred rather circuitously from the type of prob = Problem(...) (for example, if typeof(prob.eqn.calcN!) === typeof(GeophysicalFlows.TwoDimNavierStokes), then we know that prob solves the two-dimensional Navier-Stokes equations). There is only one stepforward! function for FourierFlows.Problem. I think its a valid criticism that this design is not "ideally Julian".

An alternative design would define a fundamental abstract type (such as AbstractProblem) and then ask module writers to implement a new problem type. In this design FourierFlows.jl would implement a PDE "template" by defining placeholder functions (such as calculate_nonlinear_term!(::ProblemType, args...), which module writers would extend for their PDE.

I don't think we have a great argument for taking the first approach over the second other than the knowledge of the developers at the time the code was written. At the end of the day I don't think that the first approach we currently use is that limiting even though it is imperfect julia design. Since users rarely extend GeophysicalFlows.jl functions, our design is not limiting; for example some users do indeed couple multiple physical setups together by defining appropriately different instances of FourierFlows.Problem. Because our imperfect design hasn't proved to be much of a barrier to implementing new PDEs (the difficult part is always writing calcN!, which is essentially the same task in either design for module writers), and users don't seem to be complaining about issues related to this design, we haven't been motivated to undertake a redesign of FourierFlows.jl yet.

But someday, it might be worth it.

pdebuyl commented 3 years ago

Thank you @glwagner for the extra information. I understand the appeal for being "pythonic", "julian", etc, but indeed software authors can still make architectural choices that deviate from that :-)

pdebuyl commented 3 years ago

@navidcy I can't guess what to do with the very sparse installation instructions. ] add GeophysicalFlows does not help much, can you let me know what to do. (and update the readme and also explain installation in the web documentation?)

navidcy commented 3 years ago

@navidcy I can't guess what to do with the very sparse installation instructions. ] add GeophysicalFlows does not help much, can you let me know what to do. (and update the readme and also explain installation in the web documentation?)

@pdebuyl, the docs include much more detail on how to install. https://fourierflows.github.io/GeophysicalFlowsDocumentation/dev/installation_instructions/ Could you please let me know if indeed the doc instructions seem clear to you? If so, I can update the readme to reflect what the docs include. If you still feel docs instructions are not clear then I’ll update both.

pdebuyl commented 3 years ago

Thanks for the reply. I was not familiar with the ] syntax for accessing Pkg. It does not work by copy-paste, is that normal?

I must have had a outdated cache for the documentation, the current version is fine, thanks. The readme version is then the one that didn't allow me to proceed.

I asked "add GeophysicalFlows" but still had to install explicitly FourierFlows, FFTW, and Plot. That should be clear from the doc as well.

navidcy commented 3 years ago

OK, @pdebuyl, I see... you probably tried to run the examples and run into that because, e.g., Plots.jl is not a dependency of GeophysicalFlows.jl so it must be installed separately.

We'll add a remark on that in the examples. And we'll try to clean up/smooth what's been re-exported by which package so, e.g., all FourierFlows.jl namespace is available to the user as soon they call using GeophysicalFlows.

I'll ping you here when these changes are made.

pdebuyl commented 3 years ago

Thank you @navidcy for the detailed reply. Regarding the changes that you propose, my feedback is the one of a user testing the first example. I don't know if this warrants changing the way FourierFlows and GeophysicalFlows should expose the API of imported packages. The note about installing Plots.jl is of course welcome in the example section or in the installation section. For the rest, it is important that FFTW and FourierFlows are installed as dependencies.

navidcy commented 3 years ago

@pdebuyl, I see your point.

But we discussed with @glwagner and thought that this way would feel much more natural to the GeophysicalFlows.jl users. The PR https://github.com/FourierFlows/GeophysicalFlows.jl/pull/229 adds remarks in all examples for installing packages that are not dependency of GeophysicalFlows.jl (e.g., Plots.jl) and also simplifies the packages needed to be called at the beginning by the users. I'll wait for a review before I merge. I'll ping you when it's ready for you to have a look.

navidcy commented 3 years ago

@pdebuyl, we've tagged a new release GeophysicalFlows.jl v0.12.0 which includes:

updated installation instructions in Readme
simpler loading of packages in examples (e.g., users don't need to call using FourierFlows nor using CUDA any more); see, e.g., 2D Navier-Stokes Decaying Turbulence Example
remarks on the extra packages needed to be installed for each experiment

Have a look and let me know how that looks. Note that if you've installed GeophysicalFlows.jl already you might need to update to the latest version (something that users can do following instructions in the docs).

pdebuyl commented 3 years ago

I have the error

julia> ζ₀ = peakedisotropicspectrum(grid, k₀, E₀, mask=prob.timestepper.filter)
ERROR: UndefVarError: irfft not defined
Stacktrace:
 [1] peakedisotropicspectrum(::TwoDGrid{Float64,Array{Float64,2},StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}},FFTW.cFFTWPlan{Complex{Float64},-1,false,2,UnitRange{Int64}},FFTW.rFFTWPlan{Float64,-1,false,2,UnitRange{Int64}}}, ::Int64, ::Float64; mask::Array{Float64,2}, allones::Bool) at /home/pierre/.julia/packages/GeophysicalFlows/06GdS/src/utils.jl:47
 [2] top-level scope at REPL[24]:1

I hope this is not a silly mistake on my side. I updated the code and restarted to copy-paste the first example...

pdebuyl commented 3 years ago

@whedon check references

pdebuyl commented 3 years ago

@whedon check references

pdebuyl commented 3 years ago

@whedon check references

navidcy commented 3 years ago

I have the error

julia> ζ₀ = peakedisotropicspectrum(grid, k₀, E₀, mask=prob.timestepper.filter)
ERROR: UndefVarError: irfft not defined
Stacktrace:
 [1] peakedisotropicspectrum(::TwoDGrid{Float64,Array{Float64,2},StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}},FFTW.cFFTWPlan{Complex{Float64},-1,false,2,UnitRange{Int64}},FFTW.rFFTWPlan{Float64,-1,false,2,UnitRange{Int64}}}, ::Int64, ::Float64; mask::Array{Float64,2}, allones::Bool) at /home/pierre/.julia/packages/GeophysicalFlows/06GdS/src/utils.jl:47
 [2] top-level scope at REPL[24]:1

I hope this is not a silly mistake on my side. I updated the code and restarted to copy-paste the first example...

Strange. Could you paste the output of

julia> using Pkg; Pkg.status()

?

glwagner commented 3 years ago

@navidcy do you need to update the compat tag for FourierFlows? It's set to 0.6.16, but I think the reexport stuff was added to 0.6.17?

https://github.com/FourierFlows/GeophysicalFlows.jl/blob/e4f1ba3402eaa9982b62f4775bc58680290cf1a2/Project.toml#L24

https://github.com/FourierFlows/FourierFlows.jl/pull/271/files

navidcy commented 3 years ago

🤦‍♂️

(I thought I did it...)

navidcy commented 3 years ago

@pdebuyl, I pushed a patch v0.12.1. Could you try updating again, e.g.,

julia> using Pkg; Pkg.update()

and give it a go?

If you get an error then please paste here the output from

julia> using Pkg; Pkg.status()

and

julia> versioninfo()

pdebuyl commented 3 years ago

@whedon check references

navidcy commented 3 years ago

Perhaps it needs from branch JOSS-paper?

pdebuyl commented 3 years ago

@whedon check references from branch JOSS-paper

whedon commented 3 years ago

Attempting to check references... from custom branch JOSS-paper

pdebuyl commented 3 years ago

Thanks, sometimes JOSS is unresponsive and I expected an error if it was online but my command wrong.

pdebuyl commented 3 years ago

Hi @navidcy

I could test succesfully some of the examples, thanks.

A few extra comments before proceeding:

the zenodo citation for FourierFlows in the paper lists three authors, whereas the zenodo page lists five. Can you either edit the zenodo entry or the citation so that they match?
can you cite a preprint instead for Pearson 2021? The paper does not have a pre-publication DOI and it is thus not citable as is.
the Oceananigans.jl should have a doi
minor style suggestion "to grasp on new concepts and phenomena" -> "to grasp new concepts and phenomena"

Is Version v0.12.1 10.5281/zenodo.4695260 the suitable code archive for the paper? We ask that the zenodo archive is title equally to the paper, can you edit the title to "GeophysicalFlows.jl: Solvers for geophysical fluid dynamics problems in periodic domains on CPUs & GPUs" ? The author list for the archive is not the same as the paper, I'll check in our documentation if this is an issue or not.

navidcy commented 3 years ago

Thanks. I’ll deal with these tomorrow. @BrodiePearson is there a doi for your paper yet?

navidcy commented 3 years ago

@pdebuyl, should the release that will be attached to the JOSS paper include the paper folder (e.g., merge https://github.com/FourierFlows/GeophysicalFlows.jl/tree/JOSS-paper to master and then make a release)?

pdebuyl commented 3 years ago

@navidcy the paper can remain in a separate branch, see https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements

Many authors let their paper in paper/ or doc/paper, but it is not mandatory. For publishing purposes, JOSS stores the pdf and the metadata of the paper.

BrodiePearson commented 3 years ago

@navidcy It has been assigned the following DOI: 10.1017/jfm.2021.247
@pdebuyl Because it is in the proofing stage this DOI has not been activated yet, is it still allowed? I expect it to be published imminently, and I do not have a preprint.

navidcy commented 3 years ago

I can add the DOI but I don't know if whendon will still complain since it's not active...

@BrodiePearson, any chance you push it onto arXiv so we cite that? Then, as soon as paper is published you can add the DOI and full publications details in the arXiv as metadata.

navidcy commented 3 years ago

@whedon generate pdf from branch JOSS-paper

whedon commented 3 years ago

Attempting PDF compilation from custom branch JOSS-paper. Reticulating splines etc...

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

navidcy commented 3 years ago

Hi @navidcy

I could test succesfully some of the examples, thanks.

great!

A few extra comments before proceeding:

the zenodo citation for FourierFlows in the paper lists three authors, whereas the zenodo page lists five. Can you either edit the zenodo entry or the citation so that they match?

done

the Oceananigans.jl should have a doi

done

minor style suggestion "to grasp on new concepts and phenomena" -> "to grasp new concepts and phenomena"

done

Is Version v0.12.1 10.5281/zenodo.4695260 the suitable code archive for the paper? We ask that the zenodo archive is title equally to the paper, can you edit the title to "GeophysicalFlows.jl: Solvers for geophysical fluid dynamics problems in periodic domains on CPUs & GPUs" ? The author list for the archive is not the same as the paper, I'll check in our documentation if this is an issue or not.

yes, that's a good version to associate with the JOSS paper. I've changed the Zenodo metadata to reflect the paper title and authors.

(The only outstanding remark is the citation to Pearson et al. 2021.)

pdebuyl commented 3 years ago

@navidcy thank you for the updates. Regarding the upcoming paper in J. Fluid Mech. how long until it public? If you don't want to wait I can check with the other editors how to handle such a reference.

navidcy commented 3 years ago

@navidcy thank you for the updates. Regarding the upcoming paper in J. Fluid Mech. how long until it public? If you don't want to wait I can check with the other editors how to handle such a reference.

@BrodiePearson any ETA on that?

BrodiePearson commented 3 years ago

@navidcy @pdebuyl The paper was published today, so the DOI is active [https://www.doi.org/10.1017/jfm.2021.247]. Thank you for being patient with that!

pdebuyl commented 3 years ago

Hi @BrodiePearson good news :-)

@navidcy can you update the bibliography to reflect the volume and number of the article?

pdebuyl commented 3 years ago

@whedon set v0.12.1 as version

whedon commented 3 years ago

OK. v0.12.1 is the version.

pdebuyl commented 3 years ago

@whedon set 10.5281/zenodo.4695260 as archive

whedon commented 3 years ago

OK. 10.5281/zenodo.4695260 is the archive.

navidcy commented 3 years ago

@navidcy can you update the bibliography to reflect the volume and number of the article?

Done.

openjournals / joss-reviews