[PRE REVIEW]: CASTRO: A Massively Parallel Compressible Astrophysics Simulation Code

[whedon]

Submitting author: @zingale (M. Zingale) Repository: https://github.com/amrex-astro/Castro Version: 20.07 Editor: @eloisabentivegna Reviewers: @kegiljarhus, @bonh Managing EiC: Kyle Niemeyer

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @zingale. Currently, there isn't an JOSS editor assigned to your paper.

@zingale if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Failed to discover a Statement of need section in paper

[whedon]

PDF failed to compile for issue #2468 with the following error:

pandoc-citeproc: reference ctu not found pandoc-citeproc: reference ppm not found Error producing PDF. ! Undefined control sequence. l.431 ... research. The GPU development of \castro

Looks like we failed to compile the PDF

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=1.87 s (460.8 files/s, 86976.5 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
C++                             111          11484           5015          36472
Fortran 90                      222          12858           5209          35233
Python                          110           3998           1481           9471
C/C++ Header                    103           2437           3929           7229
reStructuredText                 36           2904           1083           6994
Markdown                         33            675              0           2178
make                             81            889            198           1370
TeX                               4            134              6           1273
Bourne Shell                     64            561            241           1246
SVG                               5              5              5            850
Fortran 77                        1            353            298            719
Bourne Again Shell               71            447            752            652
INI                               2             89              0            628
Modula3                           2             83              0            300
Jupyter Notebook                  5              0           1975            161
Korn Shell                        2             76             81            107
YAML                              3             22              7            105
ANTLR Grammar                     1             19              0             62
JSON                              1              0              0             52
HTML                              3              4              5             46
CSS                               1              9              5             36
--------------------------------------------------------------------------------
SUM:                            861          37047          20290         105184
--------------------------------------------------------------------------------

Statistical information for the repository '2468' was gathered on 2020/07/10.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Abigail Bishop                   1            34              2            0.01
Alice Harpole                   43          3707            214            0.79
Andrew Myers                     6            76             59            0.03
Ann Almgren                    145         21258          16014            7.53
Brian Friesen                    1             1              1            0.00
Chris DeGrendele                 2           325              1            0.07
Chris Malone                    20           186             82            0.05
Christopher Degrende             1           329              4            0.07
Cy P Chan                        1             1              2            0.00
Donald E. Willcox                8           135             26            0.03
KiranEiden                       6          1268            304            0.32
Marc Day                         2            18              0            0.00
Maria Barrios Sazo              12          1050            124            0.24
Max Katz                      1578         48881          38275           17.60
Michael Zingale                891        165464         122355           58.13
Mike Lijewski                    2             0              4            0.00
WeiqunZhang                    537         52941          18585           14.45
blaireness                       2          1002             23            0.21
jbb                              2             4              2            0.00
jmsexton03                       2           724            185            0.18
taehoryu                         7           992            207            0.24
vince                            2           242              2            0.05

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Abigail Bishop               30           88.2         21.3                6.67
Alice Harpole              3488           94.1         14.6               13.73
Andrew Myers                 51           67.1         40.2                7.84
Ann Almgren                1506            7.1         92.5                9.63
Chris DeGrendele            325          100.0         17.0                5.85
Chris Malone                  3            1.6         95.5                0.00
Christopher Degrende          5            1.5         15.7                0.00
Cy P Chan                     1          100.0         18.5                0.00
Donald E. Willcox           116           85.9         21.3                6.03
KiranEiden                  964           76.0         16.6                9.23
Marc Day                      9           50.0         93.7               11.11
Maria Barrios Sazo          870           82.9          3.1                2.87
Max Katz                  15428           31.6         21.6                9.18
Michael Zingale           29804           18.0         20.4                9.94
WeiqunZhang               13581           25.7         56.8                8.00
blaireness                 1002          100.0         22.5               16.67
jmsexton03                  702           97.0          2.8                7.12
taehoryu                    992          100.0         34.3                3.12
vince                        71           29.3         82.2                0.00

[kyleniemeyer]

Hi @zingale, thanks for your submission!

Before we get started, I just want to check that you haven't already published a software paper describing CASTRO. (I recall seeing some papers about CASTRO in the past.) Papers using CASTRO are fine, but if you have already published a software article about CASTRO, then we would consider this to be a duplication. Exceptions would be if this represents a substantially new/different version.

[zingale]

I guess this depends on the difference between a code paper and an algorithm paper. We've mostly published on new algorithm development and performance and optimization. The only paper that covers all of Castro in general is:

• CASTRO: A New Compressible Astrophysical Solver. I. Hydrodynamics and Self-gravity, 2010, ApJ:

  https://ui.adsabs.harvard.edu/abs/2010ApJ...715.1221A/abstract

There was a paper II and III in that series that focused on radiation and neutrino transport. The code has changed a lot since 2010, and many new developers have joined that do not receive credit from citing that 2010 paper.

We've had a number of conference proceedings and focused algorithm papers:

• MAESTRO, CASTRO, and SEDONA -- Petascale Codes for Astrophysical Applications, SciDAC 2010

  https://ui.adsabs.harvard.edu/abs/2010arXiv1008.2801A/abstract

  This is quite old and doesn't go into much detail

• On the Piecewise Parabolic Method for Compressible Flow With Stellar Equations of State, ApJ, 2015

  https://ui.adsabs.harvard.edu/abs/2015ApJS..216...31Z/abstract

  This is a pure algorithm paper that focuses only on the reconstruction in the hydro for a general EOS, so is very limited in scope.

• Meeting the Challenges of Modeling Astrophysical Thermonuclear Explosions: Castro, Maestro, and the AMReX Astrophysics Suite, AstroNum 2017

  https://ui.adsabs.harvard.edu/abs/2018JPhCS1031a2024Z/abstract

  This mainly talks about applications and shows CPU / GPU scaling curves for Castro (for an early version of our GPU effort)

• Toward Resolved Simulations of Burning Fronts in Thermonuclear X-ray Bursts, AstroNum 2018

  https://iopscience.iop.org/article/10.1088/1742-6596/1225/1/012005

  This discusses Castro in the context of X-ray burst science

• The Castro AMR Simulation Code: Current and Future Developments, AstroNum 2019 (submitted)

  https://ui.adsabs.harvard.edu/abs/2019arXiv191012578Z/abstract

  This summarizes our new time-integration and GPU performance

• Improved Coupling of Hydrodynamics and Nuclear Reactions via Spectral Deferred Corrections

  https://ui.adsabs.harvard.edu/abs/2019ApJ...886..105Z/abstract

  This is an algorithm paper that describes our new time-integration strategy.

• Preparing Nuclear Astrophysics for Exascale, accepted to SC20 (should appear on astro-ph next week)

  This is our GPU paper that describes our GPU strategy.

In addition to those, there are a number of science papers that contain appendices that include some verification of a particular part of the algorithm.

The reason we are submitting to JOSS is there is no up-to-date paper to cite for the present version of the code from the current developer groups. In general, we only cite the 2010 ApJ paper in our science paper.

We've never had a paper where the reviewers looked at the source code, documentation, and verified that it ran as we describe, which is our motivation for submitting to JOSS.

[kyleniemeyer]

@zingale ok, thanks. That clears the question up for me, at least.

Hi @eloisabentivegna, can you edit this submission?

[kyleniemeyer]

@whedon invite @eloisabentivegna as editor

[whedon]

@eloisabentivegna has been invited to edit this submission.

[zingale]

it's a fine question to ask @kyleniemeyer .

[zingale]

some potential reviewers:

jramirezcr mikaem bonh kegiljarhus joshia5 fhorrobin mdelorme

[arfon]

@zingale - does this paper accompany a submission to a AAS journal, i.e. is this part of https://blog.joss.theoj.org/2018/12/a-new-collaboration-with-aas-publishing

[zingale]

no. The closest AAS paper to this is from 2010.

[eloisabentivegna]

@whedon assign @eloisabentivegna as editor

[whedon]

OK, the editor is @eloisabentivegna

[eloisabentivegna]

@whedon add @kegiljarhus as reviewer

[whedon]

OK, @kegiljarhus is now a reviewer

[eloisabentivegna]

@whedon add @bonh as reviewer

[whedon]

OK, @bonh is now a reviewer

[eloisabentivegna]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/2513.