search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
714 stars 38 forks source link

[REVIEW]: Underworld2: Python Geodynamics Modelling for Desktop, HPC and Cloud #1797

Closed whedon closed 4 years ago

whedon commented 4 years ago

Submitting author: @jmansour (john mansour) Repository: https://github.com/underworldcode/underworld2 Version: v2.9.2b Editor: @leouieda Reviewer: @gassmoeller, @gabersyd Archive: 10.5281/zenodo.3687399

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/4e96f9dba4f8f129f64770e74abdec0e"><img src="https://joss.theoj.org/papers/4e96f9dba4f8f129f64770e74abdec0e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/4e96f9dba4f8f129f64770e74abdec0e/status.svg)](https://joss.theoj.org/papers/4e96f9dba4f8f129f64770e74abdec0e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@gassmoeller & @gabersyd, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @leouieda know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @gassmoeller

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @gabersyd

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @gassmoeller, @gabersyd it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 4 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #1797 with the following error:

Can't find any papers to compile :-(

leouieda commented 4 years ago

@whedon generate pdf from branch joss

whedon commented 4 years ago 
Attempting PDF compilation from custom branch joss. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

leouieda commented 4 years ago

👋 @jmansour @gassmoeller @gabersyd this is the review thread for the paper. All of our communications will happen here from now on.

Both reviewers have checklists at the top of this thread with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#1797 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 4 weeks. Please let me know if any of you require some more time. We can also use Whedon (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me here (@leouieda) or email me privately if you have any questions/concerns.

gassmoeller commented 4 years ago

@leouieda: I have a question about the following item on the reviewer checklist:

State of the field: Do the authors describe how this software compares to other commonly-used packages?

The authors do not explicitly describe this in the paper (although they do describe their main design principle of an easily usable library, which differs somewhat from many of the other software packages in our field), but I also have two problems with this item. First it is not mentioned on the https://joss.readthedocs.io/en/latest/submitting.html#submitting-a-paper-to-joss website, and neither on the https://joss.readthedocs.io/en/latest/review_criteria.html#review-criteria website. Thus, the authors were not asked to provide such a comparison. Second, this item can be a can of worms if you want to describe it thoroughly. I think this item should be revised somewhat to either only require a short / general comparison (including mentioning this expectation on the submission website), or it should be dropped from the review checklist. Do you think this is reasonable?

jmansour commented 4 years ago

@whedon generate pdf from branch joss

whedon commented 4 years ago 
Attempting PDF compilation from custom branch joss
. Reticulating splines etc...
whedon commented 4 years ago

PDF failed to compile for issue #1797 with the following error:

error: pathspec 'joss ' did not match any file(s) known to git. Can't find any papers to compile :-(

jmansour commented 4 years ago

@whedon generate pdf from branch joss

whedon commented 4 years ago 
Attempting PDF compilation from custom branch joss. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

leouieda commented 4 years ago

I think this item should be revised somewhat to either only require a short / general comparison (including mentioning this expectation on the submission website), or it should be dropped from the review checklist. Do you think this is reasonable?

Hi @gassmoeller that is probably a new item on the checklist. I hadn't seen it before. I'll bring this up with the JOSS editors and see what we can do about it. I agree that we should be very explicit about what we want from that statement. I imagine the idea was to provide a general comparison statement, not a thorough performance/accuracy/features comparison. Thanks for pointing this out!

leouieda commented 4 years ago

On that note, @gassmoeller @gabersyd what is the status of the review? Any updates?

gassmoeller commented 4 years ago

I am satisfied with the current state of the software and paper, and think this is an important contribution. The authors have responded productively and fixed the small comments and objections I had. I am only waiting for underworldcode/underworld2#428 to be fixed (one of the authors is currently not on the paper, because he has no ORCID so far), and afterwards this is ready from my side. I particularly liked the easy distribution and online tutorials. Nice work!

gabersyd commented 4 years ago

@leouieda, I need one more week to give a complete feedback.

Gabriele Morra http://scholar.google.com/citations?sortby=pubdate&hl=en&user=DDuTppAAAAAJ&view_op=list_works , Department of Physics http://physics.louisiana.edu, School of Geosciences http://geos.louisiana.edu, University of Louisiana at Lafayette http://www.louisiana.edu

On Wed, Oct 30, 2019 at 1:45 PM Rene Gassmoeller notifications@github.com wrote:

I am satisfied with the current state of the software and paper, and think this is an important contribution. The authors have responded productively and fixed the small comments and objections I had. I am only waiting for underworldcode/underworld2#428 https://github.com/underworldcode/underworld2/issues/428 to be fixed (one of the authors is currently not on the paper, because he has no ORCID so far), and afterwards this is ready from my side. I particularly liked the easy distribution and online tutorials. Nice work!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/1797?email_source=notifications&email_token=AEZDHHYDDKSZJA6MVJIEAZLQRHI3VA5CNFSM4I7I24FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECVLOFI#issuecomment-548058901, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEZDHH2EKPPAKTLDU565ANDQRHI3VANCNFSM4I7I24FA .

jmansour commented 4 years ago

Thanks @gassmoeller & @gabersyd

jmansour commented 4 years ago

Hi @leouieda

Are the ORCID identifiers strictly necessary for submission? One of our authors is somewhat privacy concerned and would rather not sign up for an ORCID.

leouieda commented 4 years ago

@jmansour ORCIDs are encouraged but not strictly required if an author feels strongly about not providing one (our own example paper has an authors without an ORCID).

leouieda commented 4 years ago

@gassmoeller thank you for your review! :clap:

@gabersyd no worries, take your time :+1:

jmansour commented 4 years ago

Thanks @leouieda

A further question if I may. One author has since moved onto other employment and is no longer directly involved in the project. I assume that their affiliation should represent their place of employment at the time of their contribution, but possibly their current employment should be also included as means of contact. Does JOSS have any particular preference in this situation?

leouieda commented 4 years ago

@jmansour not that I'm aware of. You could always add the new affiliation and list the author as having both.

jmansour commented 4 years ago

@whedon generate pdf from branch joss

whedon commented 4 years ago 
Attempting PDF compilation from custom branch joss. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

gabersyd commented 4 years ago

@whedon commands

whedon commented 4 years ago

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references
danielskatz commented 4 years ago

👋 @gabersyd - How is this going?

31 October:

@leouieda, I need one more week to give a complete feedback.

gabersyd commented 4 years ago

A comment: the installation folder (underworld2/docs/install_guides/) contains examples only for large HPC facilities. It would be useful to offer one also for a smaller machine, so that students could try using it in a smaller machine like a powerful desktop or a small cluster that they might want to access.

I tried to follow some existing instructions like the one for Caltech fram on our Queen Bee cluster in Louisiana, but I was stuck as the system asked me to install "scons". I did not see this requirement in the instructions.

gabersyd commented 4 years ago

I finished the review. I just have an issue as I struggled with dependencies when trying to install it on our largest cluster here in Louisiana. Maybe we can look at it together, as the dependencies were not all clear to me when I tried.

Thank you Gabriele

Gabriele Morra http://scholar.google.com/citations?sortby=pubdate&hl=en&user=DDuTppAAAAAJ&view_op=list_works Associate Professor https://physics.louisiana.edu/about-us/faculty-staff Department of Physics http://physics.louisiana.edu and School of Geosciences http://geos.louisiana.edu University of Louisiana at Lafayette http://www.louisiana.edu Editor in Chief of Artificial Intelligence in Geosciences http://www.keaipublishing.com/en/journals/artificial-intelligence-in-geosciences/

On Thu, Dec 5, 2019 at 4:52 AM Daniel S. Katz notifications@github.com wrote:

👋 @gabersyd https://github.com/gabersyd - How is this going?

31 October:

@leouieda https://github.com/leouieda, I need one more week to give a complete feedback.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/1797?email_source=notifications&email_token=AEZDHH5AWNW5CYV6LPKAA6LQXDMOVA5CNFSM4I7I24FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGAKAIQ#issuecomment-562077730, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEZDHHZWAYDMHCAULGH65QTQXDMOVANCNFSM4I7I24FA .

jmansour commented 4 years ago

Thanks @gabersyd!

Note that there are updated installation instructions on the development (and joss) branch. Apologies, I thought I'd mentioned that earlier, but checking this thread I can't see mention of it. Here's the corresponding ReadTheDocs page:

https://underworld2.readthedocs.io/en/development/Installation.html

I think that should give reasonable coverage for native machine installation. More than happy to have a closer look at installation on Queen Bee so see where things are going awry.

gabersyd commented 4 years ago

Hi John, thank you for your answer. I have tried to follow all the instructions and got stuck again into the installation of scons. I am not familiar with this library. I looker at scons.org and it seems that they have mostly developed for Python 2.*, which is quite obsolete. I tried to install it manually on the cluster, and I have gotten:

UnsatisfiableError: The following specifications were found to be in conflict:

Use "conda info " to see the dependencies for each package.

Is scons so necessary for the installation of Underworld, that there is no way to get around it? Very few Python users today use Python 2.*, now that it arrive to Python 3.8.

Do you suggest me just to use Docker? I am concerned whether the performance on the cluster will be the same if I use Docker instead of a native installation. Is it just the same?

Gabriele Morra http://scholar.google.com/citations?sortby=pubdate&hl=en&user=DDuTppAAAAAJ&view_op=list_works Associate Professor https://physics.louisiana.edu/about-us/faculty-staff Department of Physics http://physics.louisiana.edu and School of Geosciences http://geos.louisiana.edu University of Louisiana at Lafayette http://www.louisiana.edu Editor in Chief of Artificial Intelligence in Geosciences http://www.keaipublishing.com/en/journals/artificial-intelligence-in-geosciences/

On Sat, Jan 4, 2020 at 5:49 PM John Mansour notifications@github.com wrote:

Thanks @gabersyd https://github.com/gabersyd!

Note that there are updated installation instructions on the development (and joss) branch. Apologies, I thought I'd mentioned that earlier, but checking this thread I can't see mention of it. Here's the corresponding ReadTheDocs page:

https://underworld2.readthedocs.io/en/development/Installation.html

I think that should give reasonable coverage for native machine installation. More than happy to have a closer look at installation on Queen Bee so see where things are going awry.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/1797?email_source=notifications&email_token=AEZDHH74XNBP3RKE7G3KVZTQ4EOBHA5CNFSM4I7I24FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIDC2IY#issuecomment-570830115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEZDHHYG7HPXGVJN75JTOKLQ4EOBHANCNFSM4I7I24FA .

jmansour commented 4 years ago

Unfortunately scons is the package we use to build Underworld, so it is necessary. There should be no issue with using it with Py3, but I don't have experience with conda. Typically, I just use pip install scons (or perhaps pip install --user scons for local install). Actually, you should simply be able to download it from the scons project page, untar, and then update your PATH environment variable so that it can locate the scons Python executable (in the untarred script directory).

https://scons.org/pages/download.html

Docker is low (Windows/OSX) or no (Linux) overhead. However, on cluster machines, there are caveats to usage, and it is generally better to use something like Singularity or Shifter which act as conduits to Docker environments in HPC environments. It really depends on how the machine is being used and you might need to chat with the system admins to see what is the best way forward. Actually there are even performance advantages to using Docker on HPC for very large Python jobs, however, if not already setup, native build will probably be the path of least resistance. I'm happy to help if I can in either case.

gabersyd commented 4 years ago

Thank you, John, I think that my review is finished. I am sure that many will use this version of Underworld as it is more user friendly than the previous version.

leouieda commented 4 years ago

@gabersyd thank you for your review! I see that all review items have been checked so I assume that you are happy to sign off on this.

SCons used to be Python 2.7 only but recent versions are Python 3.5+ only. Installing from PyPI (using pip) will get you the latest version (3.1.2), which should work. You can also install with conda from conda-forge where the latest version is 3.1.2 and should work in Python 3 (conda install scons=3.1.2 -c conda-forge). The issue is probably that the Anaconda package for SCons is outdated.

leouieda commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

PDF failed to compile for issue #1797 with the following error:

Can't find any papers to compile :-(

leouieda commented 4 years ago

@whedon generate pdf from branch joss

whedon commented 4 years ago 
Attempting PDF compilation from custom branch joss. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

whedon commented 4 years ago 
Reference check summary:

OK DOIs

- 10.21105/joss.01136 is OK
- 10.5281/zenodo.3348451 is OK
- 10.1016/j.pepi.2018.07.009 is OK
- 10.5194/se-10-969-2019 is OK

MISSING DOIs

- https://doi.org/10.1007/978-3-0348-8197-5_10 may be missing for title: Mantle Convection Modeling with Viscoelastic/Brittle Lithosphere: Numerical Methodology and Plate Tectonic Modeling
- https://doi.org/10.1016/s0021-9991(02)00031-1 may be missing for title: A Lagrangian integration point finite element method for large deformation modeling of viscoelastic geomaterials
- https://doi.org/10.1016/j.pepi.2007.06.009 may be missing for title: Computational approaches to studying non-linear dynamics of the crust and mantle 
- https://doi.org/10.1130/g39943.1 may be missing for title: Formation of cratonic lithosphere during the initiation of plate tectonics

INVALID DOIs

- None
whedon commented 4 years ago 
Reference check summary:

OK DOIs

- 10.21105/joss.01136 is OK
- 10.5281/zenodo.3348451 is OK
- 10.1016/j.pepi.2018.07.009 is OK
- 10.5194/se-10-969-2019 is OK

MISSING DOIs

- https://doi.org/10.1007/978-3-0348-8197-5_10 may be missing for title: Mantle Convection Modeling with Viscoelastic/Brittle Lithosphere: Numerical Methodology and Plate Tectonic Modeling
- https://doi.org/10.1016/s0021-9991(02)00031-1 may be missing for title: A Lagrangian integration point finite element method for large deformation modeling of viscoelastic geomaterials
- https://doi.org/10.1016/j.pepi.2007.06.009 may be missing for title: Computational approaches to studying non-linear dynamics of the crust and mantle 
- https://doi.org/10.1130/g39943.1 may be missing for title: Formation of cratonic lithosphere during the initiation of plate tectonics

INVALID DOIs

- None
leouieda commented 4 years ago

@jmansour could you please add the missing DOIs for the references outlined by Whedon above?

jmansour commented 4 years ago

@whedon generate pdf from branch joss

whedon commented 4 years ago 
Attempting PDF compilation from custom branch joss. Reticulating splines etc...
whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

jmansour commented 4 years ago

Ok, I think they should all be good now @leouieda.

And can confirm using scons 3.12 with Py3 works fine. Thanks for input @leouieda.

leouieda commented 4 years ago

@jmansour thank you for the update. I submitted a PR with some editorial fixes (mostly references and citation style). Let me know once you review and merge these changes.