[REVIEW]: HPX - The C++ Standard Library for Parallelism and Concurrency

whedon commented 4 years ago

Submitting author: @diehlpk (Patrick Diehl) Repository: https://github.com/STEllAR-GROUP/hpx Version: 1.5.0 Editor: @danielskatz Reviewer: @bhatele, @davidbeckingsale Archive: 10.6084/m9.figshare.12907034

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca"><img src="https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca/status.svg)](https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bhatele & @davidbeckingsale, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.

✨ Please try and complete your review in the next six weeks ✨

Review checklist for @bhatele

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@diehlpk) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @davidbeckingsale

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@diehlpk) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bhatele, @davidbeckingsale it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 4 years ago

PDF failed to compile for issue #2352 with the following error:

Can't find any papers to compile :-(

danielskatz commented 4 years ago

@whedon generate pdf from branch JOSS

whedon commented 4 years ago

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

danielskatz commented 4 years ago

@whedon check references from branch JOSS

whedon commented 4 years ago

Attempting to check references... from custom branch JOSS

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.5281/zenodo.598202 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/3152041.3152084 is OK

MISSING DOIs

- https://doi.org/10.1145/3337821.3337915 may be missing for title: Runtime Adaptive Task Inlining on Asynchronous Multitasking Runtime Systems
- https://doi.org/10.1109/espm2.2018.00009 may be missing for title: Asynchronous Execution of Python Code on Task-Based Runtime Systems
- https://doi.org/10.1109/mlhpc49564.2019.00009 may be missing for title: Scheduling Optimization of Parallel Linear Algebra Algorithms using Supervised Learning
- https://doi.org/10.1145/3078155.3078187 may be missing for title: Using SYCL as an implementation framework for hpx. compute
- https://doi.org/10.1109/espm2.2018.00006 may be missing for title: Integration of CUDA Processing within the C++ Library for Parallelism and Concurrency (HPX)
- https://doi.org/10.14529/jsfi150305 may be missing for title: An autonomic performance environment for exascale
- https://doi.org/10.1109/ipdrm49579.2019.00008 may be missing for title: Assessing the Performance Impact of using an Active Global Address Space in HPX: A Case for AGAS
- https://doi.org/10.1109/icppw.2009.14 may be missing for title: ParalleX: An Advanced Parallel Execution Model for Scaling-impaired Applications
- https://doi.org/10.1145/1964218.1964232 may be missing for title: Preliminary design examination of the ParalleX system from a Software and Hardware Perspective
- https://doi.org/10.1177/1094342018819744 may be missing for title: Harnessing Billions of Tasks for a Scalable Portable Hydrodynamic Simulation of the Merger of two Stars
- https://doi.org/10.1109/ipdpsw.2018.00173 may be missing for title: Methodology for Adaptive Active Message Coalescing in Task Based Runtime Systems
- https://doi.org/10.2172/1614897 may be missing for title: Implementing Software Resiliency in HPX for Extreme Scale Computing

INVALID DOIs

- http://dx.doi.org/10.1007/978-3-540-87475-1_39 is INVALID because of 'https://doi.org/' prefix

danielskatz commented 4 years ago

👋 @bhatele & @davidbeckingsale - thanks for agreeing to review this submission. Please be sure to read the comments above, and let me know if you have any questions. Basically, your job is to check the article proof and repository and check items off your checklist above. (And note that you will have to accept the invitation before you will be able to do so.)

If you see small problems that need to be discussed, feel free to discuss them here. But if you can, create a new issue in the [target repository]](https://github.com/STEllAR-GROUP/hpx) and link to this review thread in that issue to create a corresponding breadcrumb trail here.

I look forward to seeing how this review goes

danielskatz commented 4 years ago

👋 @diehlpk - note that whedon suggests changes in your references too, though whedon is not always correct, so please check and make the appropriate changes.

You can rerun the commands above to ask whedon to regenerate the pdf and check the references again as needed.

davidbeckingsale commented 4 years ago

@diehlpk - can you offer a quick bit of rationale behind author selection? A couple of contributors are not on the author list, but have more contributions (in terms of commits/loc) than some of the other authors.

diehlpk commented 4 years ago

@davidbeckingsale We sent some an email and one reminder to our mailing lists and invited people to contribute to the paper before the deadline. The current list of authors followed the invitation and contributed to the paper. However, we could not contact over 100 contributors in person.

davidbeckingsale commented 4 years ago

@davidbeckingsale We sent some an email and one reminder to our mailing lists and invited people to contribute to the paper before the deadline. The current list of authors followed the invitation and contributed to the paper. However, we could not contact over 100 contributors in person.

Perfect. Sorry for being pedantic, I am just trying to make sure that the rationale for each of the checklist points is document somewhere 😄

diehlpk commented 4 years ago

@whedon check references from branch JOSS

whedon commented 4 years ago

Attempting to check references... from custom branch JOSS

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1145/3337821.3337915 is OK
- 10.1109/espm2.2018.00009 is OK
- 10.1109/mlhpc49564.2019.00009 is OK
- 10.5281/zenodo.598202 is OK
- 10.1145/3078155.3078187 is OK
- 10.1109/espm2.2018.00006 is OK
- 10.14529/jsfi150305 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdrm49579.2019.00008 is OK
- 10.1109/icppw.2009.14 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/1964218.1964232 is OK
- 10.1007/978-3-540-87475-1_39 is OK
- 10.1109/ipdpsw.2018.00173 is OK
- 10.1145/3152041.3152084 is OK
- 10.2172/1614897 is OK

MISSING DOIs

- https://doi.org/10.1177/1094342018819744 may be missing for title: Harnessing Billions of Tasks for a Scalable Portable Hydrodynamic Simulation of the Merger of two Stars

INVALID DOIs

- None

diehlpk commented 4 years ago

@whedon generate pdf from branch JOSS

whedon commented 4 years ago

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

diehlpk commented 4 years ago

Attempting to check references... from custom branch JOSS

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

diehlpk commented 4 years ago

@whedon generate pdf from branch JOSS

whedon commented 4 years ago

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

diehlpk commented 4 years ago

@whedon check references from branch JOSS

whedon commented 4 years ago

Attempting to check references... from custom branch JOSS

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1145/3337821.3337915 is OK
- 10.1109/espm2.2018.00009 is OK
- 10.1109/mlhpc49564.2019.00009 is OK
- 10.5281/zenodo.598202 is OK
- 10.1145/3078155.3078187 is OK
- 10.1109/espm2.2018.00006 is OK
- 10.14529/jsfi150305 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdrm49579.2019.00008 is OK
- 10.1109/icppw.2009.14 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/1964218.1964232 is OK
- 10.1007/978-3-540-87475-1_39 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdpsw.2018.00173 is OK
- 10.1145/3152041.3152084 is OK
- 10.2172/1614897 is OK

MISSING DOIs

- None

INVALID DOIs

- None

diehlpk commented 4 years ago

@danielskatz We fixed all issues with the DOI reported by whedon.

diehlpk commented 4 years ago

@whedon generate pdf from branch JOSS

whedon commented 4 years ago

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

davidbeckingsale commented 4 years ago

@diehlpk - I don't feel that the paper addresses the following review point: "State of the field: Do the authors describe how this software compares to other commonly-used packages?"

Could you please add a brief overview of related work and compare hpx to these other packages?

danielskatz commented 4 years ago

👋 @diehlpk - This is just to make sure you've seen the comments and are working on responding to them...

danielskatz commented 4 years ago

👋 @bhatele - have you had a chance to start your review? Can you let us know your status?

hkaiser commented 4 years ago

👋 @diehlpk - This is just to make sure you've seen the comments and are working on responding to them...

We're working on them. Thanks.

diehlpk commented 4 years ago

@whedon check references from branch JOSS

whedon commented 4 years ago

Attempting to check references... from custom branch JOSS

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1145/3337821.3337915 is OK
- 10.1109/espm2.2018.00009 is OK
- 10.1109/mlhpc49564.2019.00009 is OK
- 10.5281/zenodo.598202 is OK
- 10.1145/3078155.3078187 is OK
- 10.1109/espm2.2018.00006 is OK
- 10.14529/jsfi150305 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdrm49579.2019.00008 is OK
- 10.1109/icppw.2009.14 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/1964218.1964232 is OK
- 10.1007/978-3-540-87475-1_39 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdpsw.2018.00173 is OK
- 10.1145/3152041.3152084 is OK
- 10.2172/1614897 is OK
- 10.1109/HPDC.2000.868632 is OK
- 10.1177/1094342007078442 is OK
- 10.1016/j.jpdc.2014.07.003 is OK
- 10.1109/SC.2012.71 is OK
- 10.1109/MCSE.2013.98 is OK

MISSING DOIs

- None

INVALID DOIs

- None

diehlpk commented 4 years ago

@whedon generate pdf from branch JOSS

whedon commented 4 years ago

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

diehlpk commented 4 years ago

@davidbeckingsale

We highlighted the review paper

Thoman, P., Dichev, K., Heller, T. et al. A taxonomy of task-based parallel programming technologies for high-performance > computing. J Supercomput 74, 1422–1434 (2018). https://doi.org/10.1007/s11227-018-2238-4

which is a very nice overview of AMTS. In addition, we added a paragraph to the paper and highlighted that the main features of HPX is the distributed computing capability with some other notable AMTS. Among this, the future-proof C++ standard conforming API is the major difference.

I am sorry for the delay, but I had two important deadlines this week.

danielskatz commented 4 years ago

👋 @bhatele & @davidbeckingsale - How are your reviews coming? Can you let us know your status?

davidbeckingsale commented 4 years ago

@danielskatz - my checklist shows the tasks I have completed. I had some other priorities last week but will make more review progress this week.

danielskatz commented 4 years ago

👋 @bhatele - When will you be able to start this?

bhatele commented 4 years ago

👋 @bhatele - have you had a chance to start your review? Can you let us know your status?

Started the review now.

diehlpk commented 4 years ago

@whedon generate pdf from branch JOSS

whedon commented 4 years ago

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

danielskatz commented 4 years ago

👋 @bhatele & @davidbeckingsale - how are things going with your reviews?

davidbeckingsale commented 4 years ago

Fine, just need to finish reviewing the actual paper. I have verified the software itself 👍

openjournals / joss-reviews