search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
694 stars 36 forks source link

[REVIEW]: t8code - modular adaptive mesh refinement in the exascale era #6887

Open editorialbot opened 2 weeks ago

editorialbot commented 2 weeks ago

Submitting author: !--author-handle-->@jmark<!--end-author-handle-- (Johannes Markert) Repository: https://github.com/DLR-AMR/t8code Branch with paper.md (empty if default branch): JOSS-paper Version: v2.0.0 Editor: !--editor-->@elbeejay<!--end-editor-- Reviewers: @jwallwork23, @DamynChipman, @alecontri Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/3d34b95792220c376553c2dcbd5bc96f"><img src="https://joss.theoj.org/papers/3d34b95792220c376553c2dcbd5bc96f/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/3d34b95792220c376553c2dcbd5bc96f/status.svg)](https://joss.theoj.org/papers/3d34b95792220c376553c2dcbd5bc96f)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@jwallwork23 & @DamynChipman & @alecontri, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @elbeejay know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @jwallwork23

📝 Checklist for @alecontri

📝 Checklist for @DamynChipman

editorialbot commented 2 weeks ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 2 weeks ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.31 s (1332.2 files/s, 409605.7 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
C++                             164           8115          16162          43952
C/C++ Header                    133           4827          18918          12338
C                                34           1405           2544           9424
YAML                             14             89            515           1449
m4                                7            298            121           1074
make                             22            214            102           1038
Markdown                         10            259              0            694
TeX                               1             60              0            477
GLSL                              6            115            323            437
CMake                             5             57              6            400
Bourne Again Shell                7             58            196            237
Bourne Shell                      6             63            216            214
Perl                              1             43             58            207
Python                            1             18            100            145
TOML                              1              1              0              5
HTML                              1              0              2              3
--------------------------------------------------------------------------------
SUM:                            413          15622          39263          72094
--------------------------------------------------------------------------------

Commit count by author:

  2752  Johannes Holke
   700  David Knapp
   666  Sandro Elsweijer
   334  Knapp
   294  ililikakis
   193  Johannes Markert
   180  jfussbro
   175  David
   160  Dreyer, Lukas
   118  Weber
   106  Ioannis Lilikakis
   100  Carsten Burstedde
    77  Niklas  Böing
    75  Chiara Hergl
    53  Veli Uenlue
    45  Alex Dutka
    44  Davknapp
    41  Zoltan
    31  holke
    22  jmark
    21  Julia Weber
    19  lukasdreyer
    12  dependabot[bot]
    11  JPGeorgeDLR
    11  Niklas
     9  Michael Schlottke-Lakemper
     7  ililikak
     6  Hendrik Ranocha
     6  Lukas Dreyer
     5  Elsweijer
     5  Prasanna Ponnusamy
     3  Andrew C Kirby
     3  Benedict Geihe
     3  Jona Ackerschott
     3  WeberJulia
     2  Evgenii Andreev
     2  Jakob Fußbroich
     2  Pierre Kestener
     2  Prasanna
     2  Schönlein, Katrin
     1  Florian Becker
     1  Jonas Thies
     1  Lilikakis
     1  Niklas997
     1  Tabea-leistikow
     1  Zoltán Csáti
     1  bgeihe
     1  schok6
editorialbot commented 2 weeks ago

Paper file info:

📄 Wordcount for paper.md is 1360

✅ The paper includes a Statement of need section

editorialbot commented 2 weeks ago

License info:

🟡 License found: GNU General Public License v2.0 (Check here for OSI approval)

editorialbot commented 2 weeks ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7034838 is OK
- 10.1137/20M1383033 is OK
- 10.1002/nme.2579 is OK
- 10.1007/978-3-642-03344-5_6 is OK
- 10.1016/j.camwa.2020.05.004 is OK
- 10.1137/15M1040049 is OK
- 10.1016/S0010-4485(00)00120-2 is OK
- 10.1016/0021-9991(84)90073-1 is OK
- 10.1145/3319797 is OK
- 10.1145/1362622.1362656 is OK
- 10.1016/j.jcp.2012.10.038 is OK
- 10.1016/S0377-0427(99)00156-9 is OK
- 10.1007/s00466-012-0736-4 is OK
- 10.1137/070681727 is OK
- 10.1137/100791634 is OK
- 10.1002/nme.397 is OK
- 10.1016/j.apnum.2004.08.040 is OK
- 10.1007/s00366-004-0295-1 is OK
- 10.1137/16M1103518 is OK
- 10.1016/j.finel.2009.06.016 is OK
- 10.1145/3401990 is OK
- 10.1016/j.cpc.2019.106866 is OK
- 10.1145/1268776.1268779 is OK
- 10.1137/0733054 is OK
- 10.1137/0715049 is OK
- 10.5281/zenodo.3996439 is OK

MISSING DOIs

- No DOI given, and none found for title: Scalable algorithms for parallel tree-based adapti...
- No DOI given, and none found for title: Industry-Relevant Implicit Large-Eddy Simulation o...
- No DOI given, and none found for title: Open CASCADE Technology
- No DOI given, and none found for title: JUWELS Supercomputer
- No DOI given, and none found for title: JUQUEEN Supercomputer
- No DOI given, and none found for title: Curved Domain Adaptive Mesh Refinement with Hexahe...
- No DOI given, and none found for title: The characteristics of 78 related airfoil sections...
- No DOI given, and none found for title: t8code: Parallel AMR on hybrid non-conforming mesh...
- No DOI given, and none found for title: The ParaView guide: Updated for ParaView version 4...
- No DOI given, and none found for title: Adaptive Verfeinerung von Prismen
- No DOI given, and none found for title: A space-filling curve for pyramidal adaptive mesh ...
- No DOI given, and none found for title: Removing hanging faces from tree-based adaptive me...
- No DOI given, and none found for title: Algorithms for tree-based adaptive meshes with inc...
- No DOI given, and none found for title: The local discontinuous galerkin method for the ad...
- No DOI given, and none found for title: Towards high-order, hybrid adaptive mesh refinemen...

INVALID DOIs

- None
editorialbot commented 2 weeks ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

elbeejay commented 2 weeks ago

Hi @jwallwork23, @DamynChipman, and @alecontri - thank you all again for offering to peer-review t8code. Hopefully the peer review process ultimately results in the creation of a software tool that is useful to the broader community.

Please see the instructions at the top of this issue on how to generate your reviewer checklist and a link to the reviewer guidelines. Do not hesitate to reach out to me if you have any questions about the JOSS review process.

Right now we are asking that reviewers try to complete their reviews in 6 weeks if possible, so by July 25 in this case. Please keep us posted here if you think you will need an extension. I will ask the bot to issue reminders in 3 weeks so we all stay aware of this on-going review.

Thanks again, Jay

elbeejay commented 2 weeks ago

@editorialbot remind @jwallwork23 in three weeks

editorialbot commented 2 weeks ago

Reminder set for @jwallwork23 in three weeks

elbeejay commented 2 weeks ago

@editorialbot remind @DamynChipman in three weeks

editorialbot commented 2 weeks ago

Reminder set for @DamynChipman in three weeks

elbeejay commented 2 weeks ago

@editorialbot remind @alecontri in three weeks

editorialbot commented 2 weeks ago

Reminder set for @alecontri in three weeks

alecontri commented 2 weeks ago

Review checklist for @alecontri

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

jwallwork23 commented 2 weeks ago

Review checklist for @jwallwork23

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

elbeejay commented 1 week ago

@danielskatz, reviewers have raised the point that it is not possible to verify the large-scale performance claims of the software without access to a large computing cluster. Do you have any guidance on how this has been handled for other JOSS submissions in the past?

Alternatively, @jmark, do you have any input on how our reviewers might test the scalability of t8code without access to exascale compute resources?

Thanks all!

danielskatz commented 1 week ago

This problem also comes up in reproducibility and artifact evaluation. One option is simply to verify what can be verified (single system performance, scaling within that system) and to say that this is all that can be verified. Perhaps a reviewer might have access to a small cluster to run some small scaling experiments provided by the author? And/or perhaps the author can provide logs and outputs of large-scale runs?

DamynChipman commented 3 days ago

I have access to a cluster I can run some small scaling tests on.

DamynChipman commented 3 days ago

Review checklist for @DamynChipman

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

jmark commented 3 days ago

@danielskatz, reviewers have raised the point that it is not possible to verify the large-scale performance claims of the software without access to a large computing cluster. Do you have any guidance on how this has been handled for other JOSS submissions in the past?

Alternatively, @jmark, do you have any input on how our reviewers might test the scalability of t8code without access to exascale compute resources?

Thanks all!

Unfortunately, not really. Only benchmarks on a cluster will show the scalability or not. I can offer to generate up-to-date benchmark data on clusters I have access to with setup parameters given by the reviewers.

danielskatz commented 2 days ago

Thanks for that offer @jmark. Also, given that @DamynChipman can run on a small cluster, this seems like it may be sufficient. @elbeejay, what do you think?

elbeejay commented 2 days ago

I agree, thanks @DamynChipman for chiming in. I'd ask that our other reviewers just make a note that they were unable to test the scalability of the package due to lack of access to a computing cluster.