search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
720 stars 38 forks source link

[REVIEW]: VlaPy: A Python package for Eulerian Vlasov-Poisson-Fokker-Planck Simulations #2182

Closed whedon closed 4 years ago

whedon commented 4 years ago

Submitting author: @joglekara (Archis Joglekar) Repository: https://github.com/joglekara/VlaPy Version: v0.1.0 Editor: @dpsanders Reviewer: @TomGoffrey, @StanczakDominik Archive: 10.5281/zenodo.4026770

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/c2b3924d7868d7bd8472c6deb011cfcc"><img src="https://joss.theoj.org/papers/c2b3924d7868d7bd8472c6deb011cfcc/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/c2b3924d7868d7bd8472c6deb011cfcc/status.svg)](https://joss.theoj.org/papers/c2b3924d7868d7bd8472c6deb011cfcc)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@TomGoffrey & @StanczakDominik, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @dpsanders know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @TomGoffrey

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @StanczakDominik

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @TomGoffrey, @StanczakDominik it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 4 years ago 
Reference check summary:

OK DOIs

- 10.1103/PhysRevE.96.043208 is OK
- 10.1088/1361-6587/aab978 is OK
- 10.1103/PhysRev.112.1456 is OK
- 10.1016/0021-9991(73)90131-9 is OK
- 10.1088/0741-3335/41/3A/001 is OK
- 10.1016/S0010-4655(02)00451-4 is OK
- 10.5281/zenodo.1238132 is OK
- 10.1038/nphys3736 is OK
- 10.1038/nphys3744 is OK
- 10.1038/nphys3745 is OK
- 10.1038/s41467-019-08435-3 is OK
- 10.5334/jors.148 is OK

MISSING DOIs

- None

INVALID DOIs

- None
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

StanczakDominik commented 4 years ago

For the "State of the field: Do the authors describe how this software compares to other commonly-used packages?" checklist item, I think this section should be sufficient:

There are many software libraries that solve the same equation set which are available in academic settings, research laboratories, and industry (e.g., [@Banks2017; @Joglekar2018]), but the community has yet to benefit from a simple-to-read, open-source Python implementation. This lack of capability is currently echoed in conversations within the PlasmaPy [@plasmapy] community (PlasmaPy is a collection of open-source plasma physics resources). Our aim with VlaPy is to take a step towards filling this need in the open-source community.

StanczakDominik commented 4 years ago
TomGoffrey commented 4 years ago

My apologies in the delay on this review.

I'm currently stalling at the statement of need and example usage requirements.

Reading the paper it's not clear to me what the primary aim of the software is. Is it intended as a research tool in its own right, an educational tool, or both? I think this could be clarified, and assuming not a purely educational tool some concrete references to example applications using 1D1V Vlasov would ideally be added.

I have a couple of other minor comments at this stage:

joglekara commented 4 years ago

Thanks for diving in @TomGoffrey. Comments inline:

My apologies in the delay on this review.

I'm currently stalling at the statement of need and example usage requirements.

Reading the paper it's not clear to me what the primary aim of the software is. Is it intended as a research tool in its own right, an educational tool, or both? I think this could be clarified, and assuming not a purely educational tool some concrete references to example applications using 1D1V Vlasov would ideally be added.

Thanks for the feedback here. The statement of need/applications is hopefully addressed in https://github.com/joglekara/VlaPy/pull/38. Looking forward to your feedback there.

I have a couple of other minor comments at this stage:

  • Is the exclusion of sphinx, sphinxcontrib-bibtex and sphinx_rtd_theme from the INSTALL_REQUIREMENTS in setup.py deliberate?

Good point. The main reason sphinx and the related packages are excluded is because we rely on the .read_the_docs.yaml and docs/requirements.txt to do the work. However, that's only configured for hosting on RTD. Do you think it be added to the setup.py for the main package too?

  • Personally I would add newline at the end of README.md and CONTRIBUTING.md, as is done in CODE_OF_CONDUCT.md

Thanks for these. The newline suggestions are also included in https://github.com/joglekara/VlaPy/pull/38

dpsanders commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

dpsanders commented 4 years ago

👋 Sorry for the delay.

@joglekara Just wanted to point out that the reference to @Cheng1977 in the paper seems to have the wrong format.

@TomGoffrey Are you happy with the updated statement of need and references and the modifications to the requirements?

@StanczakDominik I see that all of your boxes are checked off, thanks! Have you finished your review?

StanczakDominik commented 4 years ago

Oh! Yeah, I bet there's some step here that I should have done to finish it, but forgot - sorry. Getting to it after breakfast.

dpsanders commented 4 years ago

No I don't think you missed a step actually, just wanted to confirm!

TomGoffrey commented 4 years ago

Yes, the statement of need is much clearer now thanks.

Regarding the sphinx requirements discussed above, I'm afraid I don't have a definitive opinion. On one hand my guess would be that not many users would build the docs locally, but on the other hand it's not a big change and it would benefit the users who do want to build for the requirements to be added. That all being said as I don't have a strong opinion I'm happy to leave the decision to you.

Some additional comments:

  1. I think the paper itself would benefit from a plot (or two) for the final section. I think you could easily adapt the later plots from the notebook? Obviously with the notebook the user could get the plots that way, but I think my adding it to the paper you make it a stronger stand-alone document.
  2. The other tests are mentioned, but it's not obvious how to run them. Obviously it doesn't take much searching through the test directory, but a simple comment along the lines of these tests are demonstrated in tests/test_lb.py would be beneficial. Even better (but I guess more work) would be (e.g.) a plot showing the relaxation to a Maxwellian. Again, sounds basic but in my experience many first time users like to be able to verify they have run the code correctly, and regenerating published plots is an obvious starting point.
  3. Is there a reason many of the RTD pages end in '…this page is under development…'? Is there more to do?

It's possible some (all?) of the above is considered beyond the required scope of the article, as I think I could reasonably sign off on the checklist now. @dpsanders could you comment perhaps?

dpsanders commented 4 years ago

Thanks @TomGoffrey! My opinion is that simple modifications to the paper and tests that would make the paper and repo more accessible should definitely be done.

joglekara commented 4 years ago

Thanks @TomGoffrey for the suggestions!

Yes, the statement of need is much clearer now thanks.

Regarding the sphinx requirements discussed above, I'm afraid I don't have a definitive opinion. On one hand my guess would be that not many users would build the docs locally, but on the other hand it's not a big change and it would benefit the users who do want to build for the requirements to be added. That all being said as I don't have a strong opinion I'm happy to leave the decision to you.

Some additional comments:

  1. I think the paper itself would benefit from a plot (or two) for the final section. I think you could easily adapt the later plots from the notebook? Obviously with the notebook the user could get the plots that way, but I think my adding it to the paper you make it a stronger stand-alone document.

Agreed! I will implement that.

  1. The other tests are mentioned, but it's not obvious how to run them. Obviously it doesn't take much searching through the test directory, but a simple comment along the lines of these tests are demonstrated in tests/test_lb.py would be beneficial. Even better (but I guess more work) would be (e.g.) a plot showing the relaxation to a Maxwellian. Again, sounds basic but in my experience many first time users like to be able to verify they have run the code correctly, and regenerating published plots is an obvious starting point.

Good idea. Implementing this in a Jupyter Notebook seems like the best way forward. Do you agree?

  1. Is there a reason many of the RTD pages end in '…this page is under development…'? Is there more to do?

Well, there's always more to do ;). For many of those notes, I was referring to the fact that the the background to some of the finite differencing or kinetic equation derivations is still a work in progress.

Perhaps it is better to clean up any loose ends, trim and close the loop? Or leave the loop open while explicitly stating what is still to be done? @TomGoffrey @StanczakDominik @dpsanders , your thoughts are welcome here.

It's possible some (all?) of the above is considered beyond the required scope of the article, as I think I could reasonably sign off on the checklist now. @dpsanders could you comment perhaps?

TomGoffrey commented 4 years ago

@joglekara I do agree that a notebook would be the best way, looks like you're already working on it. That's great.

I agree, documentation is always WIP, but (and I guess this makes it more your decision than mine) I think the code looks better to (prospective) users if you tie up any loose ends and remove the comments. You can always expand on details later. That's just my opinion though, so as you say if others want to weigh in that would be good.

joglekara commented 4 years ago

I think we might be close...

@dpsanders, @TomGoffrey , @StanczakDominik , any other thoughts or comments to improve this even further?

TomGoffrey commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

PDF failed to compile for issue #2182 with the following error:

Error producing PDF. ! Missing $ inserted.

$ l.484 where \$v\_0 = \int Looks like we failed to compile the PDF
joglekara commented 4 years ago

@whedon commands

whedon commented 4 years ago

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository
joglekara commented 4 years ago

@whedon generate pdf from branch paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #2182 with the following error:

error: pathspec 'paper' did not match any file(s) known to git. Error producing PDF. ! Missing $ inserted.

$ l.484 where \$v\_0 = \int Looks like we failed to compile the PDF
joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #2182 with the following error:

Error producing PDF. ! Missing $ inserted.

$ l.484 where \$v\_0 = \int Looks like we failed to compile the PDF
joglekara commented 4 years ago

Sorry all, please let me know if you have a better way of debugging this!

joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper
. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #2182 with the following error:

error: pathspec 'origin/paper ' did not match any file(s) known to git. error: pathspec 'origin/paper ' did not match any file(s) known to git. Error producing PDF. ! Missing $ inserted.

$ l.484 where \$v\_0 = \int Looks like we failed to compile the PDF
joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #2182 with the following error:

Error producing PDF. ! Missing $ inserted.

$ l.491 distribution and \$v\_\{t\} = \int Looks like we failed to compile the PDF
dpsanders commented 4 years ago

At a guess, try removing the comma and space at the end of $v_0 = \int v^2 f(x,v) dv, $. The procedure of parsing the LaTeX equations out of the markdown source is delicate.

Also cc-ing @arfon since

@whedon generate pdf from branch paper

didn't seem to work

dpsanders commented 4 years ago

Also @arfon, is there an easy way of compiling the PDF locally to iron out these things?

joglekara commented 4 years ago

I actually think it's unable to render $\int$ inline. It seems to move on to the next inline $\int$ call and get stuck there when i remove the first one.

I'm pushing a version with all integrals in their $$ blocks

joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

dpsanders commented 4 years ago

I suggest using vector graphics (PDF) in the figures rather than PNG, if possible, so that the resolution is higher.

joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

arfon commented 4 years ago

Also @arfon, is there an easy way of compiling the PDF locally to iron out these things?

Authors can use the Whedon preview service at http://whedon.theoj.org

joglekara commented 4 years ago

@whedon generate pdf from branch origin/paper

whedon commented 4 years ago 
Attempting PDF compilation from custom branch origin/paper. Reticulating splines etc...