[PRE REVIEW]: PyRevolve: A python library to manage checkpointing for adjoints

[whedon]

Submitting author: @navjotk (Navjot Kukreja) Repository: https://github.com/devitocodes/pyrevolve Version: v2.2 Editor: Pending Reviewer: Pending Managing EiC: Lorena A Barba

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @navjotk. Currently, there isn't an JOSS editor assigned to your paper.

@navjotk if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.13 s (319.8 files/s, 54898.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          19            612           1070           1860
C++                              2            146            156           1317
SVG                              9              9              9            644
C/C++ Header                     2            117            138            246
JavaScript                       1             30              3            199
Cython                           2             30             14            140
YAML                             2             11              0            106
CSS                              1             10              0             70
Markdown                         2             18              0             58
HTML                             1              2              0             23
-------------------------------------------------------------------------------
SUM:                            41            985           1390           4663
-------------------------------------------------------------------------------

Statistical information for the repository '2487' was gathered on 2020/07/18.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Jan Hueckelheim                 12          5554           3135           63.02
Michael Lange                    1             2              1            0.02
Navjot Kukreja                  48          1452            607           14.93
Tim Burgess                      1             2              2            0.03
navjotk                         22          2830            202           21.99

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Jan                        2317          100.0         33.3               18.69
Michael Lange                 2          100.0         25.9                0.00
Navjot Kukreja             3573          246.1         17.7               12.09
Tim Burgess                   2          100.0         23.4                0.00

[whedon]

PDF failed to compile for issue #2487 with the following error:

Can't find any papers to compile :-(

[labarba]

@whedon generate pdf from branch joss_paper

[whedon]

Attempting PDF compilation from custom branch joss_paper. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[labarba]

@whedon check references

[labarba]

👋 @navjotk -— We're going to need DOIs added to all references that have them. Go ahead and make those edits, then run the command @whedon generate pdf from branch joss_paper here, as a single-line comment.

[labarba]

@whedon invite @jedbrown as editor

[whedon]

@jedbrown has been invited to edit this submission.

[labarba]

@whedon check references from branch joss_paper

[whedon]

Attempting to check references... from custom branch joss_paper

[whedon]

Reference check summary:

OK DOIs

- None

MISSING DOIs

- https://doi.org/10.1007/978-3-030-29400-7_7 may be missing for title: Combining checkpointing and data compression to accelerate adjoint-based optimization problems
- https://doi.org/10.1109/tpds.2020.2982626 may be missing for title: An Event-Driven Approach to Serverless Seismic Imaging in the Cloud
- https://doi.org/10.1190/geo2018-0490.1 may be missing for title: Compressive least-squares migration with on-the-fly Fourier transforms
- https://doi.org/10.1190/geo2018-0174.1 may be missing for title: A large-scale framework for symbolic implementations of seismic inversion algorithms in Julia
- https://doi.org/10.5194/gmd-12-1165-2019 may be missing for title: Devito (v3. 1.0): an embedded domain-specific language for finite differences and geophysical exploration
- https://doi.org/10.1109/tvcg.2014.2346458 may be missing for title: Fixed-rate compressed floating-point arrays
- https://doi.org/10.1109/ipdps.2016.11 may be missing for title: Fast error-bounded lossy HPC data compression with SZ
- https://doi.org/10.1145/347837.347846 may be missing for title: Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation
- https://doi.org/10.21105/joss.01292 may be missing for title: dolfin-adjoint 2018.1: automated adjoints for FEniCS and Firedrake

INVALID DOIs

- None

[jedbrown]

@navjotk Thanks for your submission. In looking it over, I'm not sure this is appropriate for JOSS in its current form: https://joss.readthedocs.io/en/latest/submitting.html#substantial-scholarly-effort

In particular, I understand that src/revolve.cpp is preexisting software (the link is 404, BTW). If we subtract out versioneer, I'm seeing less than 500 LoC in pyadjoint/, plus the Cython that is solely wrapping the C/C++ library. I see that it significantly improves the ergonomics of using Revolve, but this still feels like a wrapper more than a substantial product in its own right. There has been a long-term refinement of these criteria at JOSS and I'd like to give you a chance to comment and self-evaluate before discussing with the Editorial Board.

I'm also concerned that the package isn't pip installable. I know it's a bit of a hassle with Cython, but many other packages do it and it greatly lowers effort for prospective users to experiment.

[labarba]

@whedon query scope

[whedon]

Submission flagged for editorial review.

[navjotk]

@whedon generate pdf from branch joss_paper

[whedon]

Attempting PDF compilation from custom branch joss_paper
. Reticulating splines etc...

[whedon]

PDF failed to compile for issue #2487 with the following error:

error: pathspec 'joss_paper ' did not match any file(s) known to git. Can't find any papers to compile :-(

[navjotk]

Hi @labarba . Thanks for that. I updated the references to contain DOIs (included them in the bib latex doi field). Can you please help me figure out what I did wrong in my command to whedon above? :(

[labarba]

@openjournals/dev 👋 Can you have a look at the error compiling this paper?

[labarba]

@navjotk -- Please respond to the enquiry from our editor, @jedbrown, above https://github.com/openjournals/joss-reviews/issues/2487#issuecomment-660658244

[navjotk]

Hi @jedbrown Thank you very much for your feedback. PyRevolve is indeed pip installable and on PyPI. I have updated the README to make this stand out more. I have also fixed the broken link to Revolve.

A novel contribution in PyRevolve is the inclusion of lossy compression in the API. PyRevolve currently supports Blosc and ZFP, with support for SZ coming soon. We are using this package as part of ongoing research into the impact of lossy compression in back propagation for PDE-constrained optimisation problems. Here is a paper from last year based on this. We have another one coming out soon.

[jedbrown]

Thanks @navjotk; good to hear it's on PyPI. We'll get back to you in a few days regarding the scope query.

[xuanxu]

@whedon generate pdf from branch joss_paper

[whedon]

Attempting PDF compilation from custom branch joss_paper. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arfon]

@navjotk - many thanks for your submission to JOSS. After consulting with the editorial team, we have concluded that this software is out of scope for JOSS as a minimal library wrapping other functionality.

[arfon]

@whedon reject

[whedon]

Paper rejected.