[REVIEW]: SPbLA: The Library of GPGPU-powered Sparse Boolean Linear Algebra Operations

whedon commented 3 years ago

Submitting author: !--author-handle-->@EgorOrachyov@Nikoleta-v3<!--end-editor-- Reviewers: @mlxd, @bencardoen Archive: 10.5281/zenodo.7009938

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/1da10be51ab03b0f574f2fda4aabc3f9"><img src="https://joss.theoj.org/papers/1da10be51ab03b0f574f2fda4aabc3f9/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/1da10be51ab03b0f574f2fda4aabc3f9/status.svg)](https://joss.theoj.org/papers/1da10be51ab03b0f574f2fda4aabc3f9)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@abb58 & @mlxd, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @Nikoleta-v3 know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @abb58

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[ ] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@EgorOrachyov) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @mlxd

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@EgorOrachyov) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Nikoleta-v3 commented 2 years ago

@editorialbot add @bencardoen to reviewers

editorialbot commented 2 years ago

@bencardoen added to the reviewers list!

Nikoleta-v3 commented 2 years ago

Thank you very much @bencardoen for agreeing to review this submission! 🙏🏻

this is my first JOSS review

This is great - welcome! 🎉

If you comment the following command @editorialbot generate my checklist on this issue the bot will generate a checklist for you ✍🏻

What you have to do is go over the checklist and make sure that the submission addresses each point. These points will include details both about the software and the paper.

I recommend that each time you would like the authors to change something that you open an issue on their repository (https://github.com/JetBrains-Research/spbla) and in there mention this issue (#3743). That way I can keep track of the issues!

Once you are done with the checklist you can just leave a comment here to let me know. If you have any questions regarding something on the checklist (or anything else) you can ping me here and I will reply as soon as possible!

bencardoen commented 2 years ago

Review checklist for @bencardoen

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/JetBrains-Research/spbla?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@EgorOrachyov) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

bencardoen commented 2 years ago

@Nikoleta-v3 I completed the checklist, and made issues (I hope in the right way) for the items I came across. My general review comments are in issue https://github.com/JetBrains-Research/spbla/issues/19 I haven't been able to verify the performance results due to a build issue https://github.com/JetBrains-Research/spbla/issues/17 so that's unchecked, and I still need to get the python tests running. I'll revisit this comment thread once I've completed the Python part.

Nikoleta-v3 commented 2 years ago

That's great, thank you for the update @bencardoen 😄

The comments regarding the paper look good and addressable 👍🏻 I hope the build issue is resolved soon.

Nikoleta-v3 commented 2 years ago

👋🏻 @bencardoen any updates? 😄

bencardoen commented 2 years ago

@Nikoleta-v3 https://github.com/JetBrains-Research/spbla/issues/19 (general feedback/comments) is open, the other issues have been addressed. I've updated the comment above to reflect the changes.

Nikoleta-v3 commented 2 years ago

Perfect -- thank you! @EgorOrachyov ⬆️ please have a look at the issue.

Nikoleta-v3 commented 2 years ago

@EgorOrachyov any updates on this?

EgorOrachyov commented 2 years ago

@Nikoleta-v3 Hi! Yes, we have addressed some minor technical and documentation related features. Now we are going to do a text clean-up, as it is suggested in https://github.com/JetBrains-Research/spbla/issues/19 issue. I hope, we will have it done in a couple of days.

Apologies for delays, busy work days :/

Nikoleta-v3 commented 2 years ago

Thank you for the update!

editorialbot commented 2 years ago

My name is now @editorialbot

editorialbot commented 2 years ago

My name is now @editorialbot

EgorOrachyov commented 2 years ago

@Nikoleta-v3 Hi, I hope, we have addressed all issues. Waiting for a feedback from @bencardoen

It would be great, if you trigger paper recompilation (seems I do not have enough permissions to do it), since we updates text and some diagrams :)

@whedon generate pdf from branch joss-paper

Nikoleta-v3 commented 2 years ago

@editorialbot generate pdf from branch joss-paper

editorialbot commented 2 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

Nikoleta-v3 commented 2 years ago

@editorialbot set joss-paper as branch

editorialbot commented 2 years ago

Done! branch is now joss-paper

Nikoleta-v3 commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

bencardoen commented 2 years ago

@Nikoleta-v3 @EgorOrachyov I'm ok with the changes made in issue 19, however noted the below writing issues (quickly fixed, so just listing them here if that's ok)

Line 10: within [a] Python runtime Line 48 Related Tools: Best to our knowledge --> To the best of our knowledge Line 74 Experiment is --> The experiment is
Line 86 Future research : Potential solution --> A potential solution

Apart from those minor issues I'm satisfied. (Thank you @EgorOrachyov for the clarifying answer to my question in the issue)

EgorOrachyov commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

EgorOrachyov commented 2 years ago

@bencardoen Hi, thank you once more 😀! I've fixed minor writing issues, new pdf file is above.

bencardoen commented 2 years ago

@EgorOrachyov thanks for the quick turnaround, latest version of the paper looks great. @Nikoleta-v3 Do I need to issue a special command to conclude the review on my part?

Nikoleta-v3 commented 2 years ago

Hey @bencardoen thank you so much for your review! Apologies for my late reply (I am currently at a workshop 💻 ).

No, I can see that you have ticked all the boxes on your list so that's all 😄 Thank you for your time once again 🙏🏻

@EgorOrachyov - I already have looked over the source code. I will have one final look at the paper at some point this week and see if there are any more minor things that we might have missed and then we should be good to go!

EgorOrachyov commented 2 years ago

@Nikoleta-v3 Hi, thank you! We will be waiting for a feedback from you.

Nikoleta-v3 commented 2 years ago

@EgorOrachyov thank you for your patience. There are a few typos in the paper. I believe I've picked them all up (see below).

Another minor comment I have is regarding the figures. The numbers (the memory consumption) are not always easy to see. Could you maybe use a bold font?

Typos

Summary

as a stand-alone self-sufficient

Statement of need

to leverage a set of ~a~ powerful sparse linear algebra libraries (I think)
Also, we hope ~,~ that the library is a small step to move the implementation (remove second comma)

Related tools

is one of the first standards ~,~ that formalize the mathematical (remove comma)
in the language of ~the~ linear algebra.
is a reference implementation~s~ of the (I think)
It is a mature and fully featured library with a number of bindings for
is currently in ~the~ active development.
There are a number of open-source and proprietary libraries

Performance

The experiment itself is designed as a computational task~s~, which arises as a stand-alone
memory consumption in some cases, ~what~ which can be a critical limitation

Future research

in order to support arbitrary types and operations, defined by the user.

Nikoleta-v3 commented 2 years ago

@EgorOrachyov 🆙 👋🏻 😄

Nikoleta-v3 commented 2 years ago

@EgorOrachyov sorry for the multiple pings but it would be nice to see this paper published soon.

EgorOrachyov commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

EgorOrachyov commented 2 years ago

@Nikoleta-v3

Hello! I apologize for such a long response. We have corrected typos and updated the charts, highlighting the measurements numbers in bold.

I hope to proceed without delay to carry out the rest of the work 👽

Nikoleta-v3 commented 2 years ago

Thank you @EgorOrachyov!

At this point could you:

[ ] Let me know which tagged release of your software we should use? v1.0.0? or would you like to make a new one?
[ ] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
[ ] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[ ] Please list the DOI of the archived version here.

I can then move forward with accepting the submission.

EgorOrachyov commented 2 years ago

@Nikoleta-v3

[x] Release v1.0.0 is ok. (I have made new one especially for Zenodo v1.0.0-zenodo, but it is effectively the same thing)
[x] Zenodo archive DOI: 10.5281/zenodo.7009938
[x] Meta filled, orcid added where possible
[x] DOI: 10.5281/zenodo.7009938

Nikoleta-v3 commented 2 years ago

@editorialbot set 10.5281/zenodo.7009938 as archive

editorialbot commented 2 years ago

Done! Archive is now 10.5281/zenodo.7009938

Nikoleta-v3 commented 2 years ago

@editorialbot set v1.0.0 as version

editorialbot commented 2 years ago

Done! version is now v1.0.0

Nikoleta-v3 commented 2 years ago

@editorialbot recommend-accept

editorialbot commented 2 years ago

Attempting dry run of processing paper acceptance...

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/978-3-319-46523-4_38 is OK
- 10.1186/1471-2105-14-149 is OK
- 10.1145/2499370.2462159 is OK
- 10.1145/3210259.3210264 is OK
- 10.1007/978-3-030-54832-2_6 is OK
- 10.1145/3322125 is OK
- 10.1016/j.jpdc.2015.06.010 is OK
- 10.1145/2909437.2909442 is OK
- 10.1109/HPEC.2016.7761646 is OK

MISSING DOIs

- 10.1145/3466795 may be a valid DOI for title: GraphBLAST: A High-Performance Linear Algebra-based Graph Framework on the GPU

INVALID DOIs

- None

editorialbot commented 2 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/3448, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

Nikoleta-v3 commented 2 years ago

@EgorOrachyov there is a missing DOI. Could you please take care of this? 😄

Nikoleta-v3 commented 2 years ago

My apologises for only raising the issue now. I was sure I checked the references 😅

EgorOrachyov commented 2 years ago

@Nikoleta-v3 If I understand you correctly, I have to fix doi 10.1145/3466795 for GraphBLATS paper and updated article sources?

EgorOrachyov commented 2 years ago

@editorialbot generate pdf

EgorOrachyov commented 2 years ago

@Nikoleta-v3 Seems to be fixed now

@EgorOrachyov there is a missing DOI. Could you please take care of this? 😄

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

openjournals / joss-reviews