[PRE REVIEW]: eigentools: A Python package for studying differential eigenvalue problems with an emphasis on robustness

whedon commented 3 years ago

Submitting author: @jsoishi (Jeffrey S. Oishi) Repository: https://github.com/DedalusProject/eigentools Version: v 2.2101 Editor: @dpsanders Reviewers: @ketch, @caropen Managing EiC: Arfon Smith

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @jsoishi. Currently, there isn't an JOSS editor assigned to your paper.

The author's suggestion for the handling editor is @jedbrown.

@jsoishi if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.47 s (36.2 files/s, 6203.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          11            344            564            881
Jupyter Notebook                 2              0            820            180
Markdown                         1             13              0             35
reStructuredText                 1              9              3             26
YAML                             1              6              0             16
make                             1              4              6             10
-------------------------------------------------------------------------------
SUM:                            17            376           1393           1148
-------------------------------------------------------------------------------

Statistical information for the repository '8bac670910e225342b0ec1e4' was
gathered on 2021/01/15.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Benjamin Brown                   1             1              3            0.07
Evan Anders                     39          1003            482           24.72
J. S. Oishi                     55          1803           1521           55.34
J.S. Oishi                      25           695             87           13.02
Keaton J. Burns                  1             3              0            0.05
Susan Clark                      2           397             12            6.81

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Evan Anders                 148           14.8         40.5               16.89
J. S. Oishi                1595           88.5          5.8               11.35
Susan Clark                  46           11.6         58.3                8.70

arfon commented 3 years ago

@whedon query scope

@jsoishi - due to the small size of this code, the editors will now discuss if it meets the substantial scholarly effort criterion for review by JOSS. We should get back to you sometime next week.

whedon commented 3 years ago

Submission flagged for editorial review.

whedon commented 3 years ago

PDF failed to compile for issue #2963 with the following error:

Can't find any papers to compile :-(

arfon commented 3 years ago

@whedon generate pdf from branch joss_paper

whedon commented 3 years ago

Attempting PDF compilation from custom branch joss_paper. Reticulating splines etc...

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jedbrown commented 3 years ago

I'm conflicted. I'd mention that Elemental (written by JOSS editor @poulson) has support for pseudospectra.

kthyng commented 3 years ago

Hi @jsoishi and thanks for your submission. The overwhelming comment from the editorial board discussing whether this was in scope or not was "why isn't this part of Dedalus?"

jsoishi commented 3 years ago

Hi @kthyng,

Thanks for considering the paper. We feel that this question is best answered by considering the scope of the two packages. Dedalus is dedicated to the spectral discretization and solution of PDEs. It provides basic solvers but intentionally does not implement high-level workflows that are not inherently tied to the spectral discretization. For example, it does not do any degree of spatial refinement or error checking of solutions. It does not include higher level functionality like adjoint looping for PDE optimization, rather just the basic solvers for implementing all of these tasks.

Eigentools focuses on implementing well-tested algorithms for error estimation, automated parameter space exploration and optimization, tools for plotting, and other high-level functionality specific to eigenvalue problems. While it currently interfaces with Dedalus, nothing in eigentools inherently requires spectral discretization. The tools for eigenvalue problems provided by eigentools are therefore not within the scope of the core Dedalus code.

As a similar example of separating scope, the dolfin-adjoint solver (published in JOSS) constructs adjoint solvers for previously derived FENICS PDE models. Adjoint methods are also related to the pseudospectra implemented by eigentools, as they provide similar information about the underlying PDE models.

Separately, having read the substantial scholarly effort, we feel that we meet all of the criteria:

eigentools is five years old with substantial development effort over that time (first commit 15 Jan 2016)
eigentools has 146 commits by 5 authors
eigentools has 1148 SLOC and 1393 comment lines; a core development philosophy is code compactness
eigentools has been cited by at least 7 papers, including at least one by an author unconnected to the development team
eigentools is in active use by several research groups world wide; once a JOSS paper appears, these groups are highly likely to cite it
eigentools is feature-complete; it implements algorithms distinct from those provided by Dedalus
eigentools is not a single-function package, nor an API client.

We hope this information makes clear why we developed eigentools in this way and why we believe it is appropriate for publication in JOSS.

Sincerely,

@jsoishi on behalf of the authors

kthyng commented 3 years ago

Hi @jsoishi, thanks for your response. The editorial board recommends moving forward to review on this submission. One additional note to leave you with was that the software could be more useful if it supported models defined outside of Dedalus (similar to how scipy.integrate is general-purpose).

kthyng commented 3 years ago

@dpsanders could you edit this submission?

kthyng commented 3 years ago

@whedon invite @dpsanders as editor

whedon commented 3 years ago

@dpsanders has been invited to edit this submission.

dpsanders commented 3 years ago

@kthyng Sure, happy to.

dpsanders commented 3 years ago

@whedon add @dpsanders as editor

whedon commented 3 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

dpsanders commented 3 years ago

@whedon commands

whedon commented 3 years ago

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Re-invite a reviewer (if they can't update checklists)
@whedon re-invite @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# All commands can be run on a non-default branch, to do this pass a custom 
# branch name by following the command with `from branch custom-branch-name`.
# For example:

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Remind an author or reviewer to return to a review after a
# certain period of time (supported units days and weeks)
@whedon remind @reviewer in 2 weeks

# Ask Whedon to do a dry run of accepting the paper and depositing with Crossref
@whedon accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

EiC TASKS

# Invite an editor to edit a submission (sending them an email)
@whedon invite @editor as editor

# Reject a paper
@whedon reject

# Withdraw a paper
@whedon withdraw

# Ask Whedon to actually accept the paper and deposit with Crossref
@whedon accept deposit=true

dpsanders commented 3 years ago

@whedon assign @dpsanders as editor

whedon commented 3 years ago

OK, the editor is @dpsanders

dpsanders commented 3 years ago

👋 @jsoishi: Thanks for submitting your work to JOSS!

My first initial observations:

I don't see any narrative documentation at all that explains how to use the package.
As far as I can tell from the README, eigentools requires dedalus to be installed. In that case, pip install eigentools should automatically install dedalus.
Could you please address @kthyng's comment that the package should ideally work beyond the scope of dedalus.

Thanks!

jsoishi commented 3 years ago

Thanks @dpsanders for your comments.

First, we have updated the documentation to provide specific links to Jupyter notebooks outlining all the features of eigentools. You can see those efforts in either the main branch of the repository or on readthedocs.

Second, Dedalus is already in requirements.txt for eigentools, so pip will attempt to install it. However, because Dedalus itself requires some dependencies that are not pip installable, we have added a warning that without those dependencies the Dedalus pip install will fail. There are detailed instructions and an automated script to install them all via conda on the Dedalus website.

Third, the expansion of eigentools to support matrices produced by other PDE solvers is an interesting issue. While there are no mathematical issues to overcome, there are significant API changes that would need to be implemented in order to create a wrapper level to translate between the quantities needed by eigentools (e.g. eigenvalues and eigenvectors in certain order, grid point values) and the methods required of the external eigenvalue solvers (e.g. parameter inputs, copy constructors for creating higher resolution problems for eigenvalue rejection). This work is an interesting future direction, but currently, eigentools users haven't yet requested support for other solvers. When that time comes, we would welcome contributions and would be happy to work with developers from other PDE solving packages interested in adapting eigentools for their solvers. We would be happy to address this as future work in the JOSS manuscript if the editors or referees would find that useful.

thanks, @jsoishi on behalf of the authors

dpsanders commented 3 years ago

Thanks, @jsoishi.

Do you have any suggestions for possible reviewers, from the list linked or otherwise?

jsoishi commented 3 years ago

hi @dpsanders,

Here are a few:

@highlando @edoddridge @ketch @sivaramambikasaran @MakrinaAgaoglou

thanks,

@jsoishi

dpsanders commented 3 years ago

👋 @ketch and @highlando: Would you be able and willing to review this submission for JOSS?

dpsanders commented 3 years ago

@jsoishi: Apologies for the delay; thanks for the suggestions.

ketch commented 3 years ago

Yes, I can do that.

highlando commented 3 years ago

I would like to suggest my colleague Carolin Penke @caropen for that. She has a proven expertise on solving PDE eigenvalue problems.

dpsanders commented 3 years ago

👋 @caropen: Would you be available to review this submission for JOSS, the Journal of Open Source Software (https://joss.theoj.org)?

Thanks for the suggest, @highlando!

caropen commented 3 years ago

Yes, I'm available for reviewing. Will you send me further instructions?

dpsanders commented 3 years ago

@whedon add @ketch as reviewer

whedon commented 3 years ago

OK, @ketch is now a reviewer

dpsanders commented 3 years ago

@whedon add @caropen as reviewer

whedon commented 3 years ago

OK, @caropen is now a reviewer

dpsanders commented 3 years ago

@whedon start review

whedon commented 3 years ago

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/3079.

dpsanders commented 3 years ago

Thanks @ketch and @caropen!

You should have been invited to the review issue, where there is a checklist for the review and links to the reviewer guidelines.

caropen commented 3 years ago

@whedon generate pdf from branch joss_paper

whedon commented 3 years ago

Attempting PDF compilation from custom branch joss_paper. Reticulating splines etc...

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

openjournals / joss-reviews

[PRE REVIEW]: eigentools: A Python package for studying differential eigenvalue problems with an emphasis on robustness #2963