editorialbot commented 2 months ago

Submitting author: !--author-handle-->@espottesmith@mbarzegary<!--end-editor-- Reviewers: @Anshuman5, @ptmerz, @lorenzo-rovigatti Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/7d071341934be452863d5523edb17413"><img src="https://joss.theoj.org/papers/7d071341934be452863d5523edb17413/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/7d071341934be452863d5523edb17413/status.svg)](https://joss.theoj.org/papers/7d071341934be452863d5523edb17413)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@Anshuman5 & @ptmerz & @lorenzo-rovigatti, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @mbarzegary know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @Anshuman5

📝 Checklist for @lorenzo-rovigatti

📝 Checklist for @ptmerz

editorialbot commented 2 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 2 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.34 s (246.9 files/s, 39609.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                             26           1091            558           4470
C/C++ Header                    27            618            354           2236
Jupyter Notebook                 5              0           1178           1479
SVG                              1              0              0            308
make                             4             86             85            162
Markdown                         3             31              0            147
TeX                              1             11              0            114
Ruby                             1             28             12            106
Bourne Shell                     2             23              2             71
Nix                              2              8              4             63
YAML                             2              1              4             34
JSON                             9              0              0             32
-------------------------------------------------------------------------------
SUM:                            83           1897           2197           9222
-------------------------------------------------------------------------------

Commit count by author:

   229  Laura Zichi
   174  Daniel Barter
    28  Evan Walter Clark Spotte-Smith
    28  lzichi
    16  Evan Walter Clark Spotte-Smith, PhD

editorialbot commented 2 months ago

Paper file info:

📄 Wordcount for paper.md is 964

✅ The paper includes a Statement of need section

editorialbot commented 2 months ago

License info:

🟡 License found: Other (Check here for OSI approval)

editorialbot commented 2 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1021/acs.nanolett.3c01955 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Exact stochastic simulation of coupled chemical re...
- No DOI given, and none found for title: Crossing the mesoscale no-mans land via parallel k...
- No DOI given, and none found for title: Electrochemical Systems

❌ MISSING DOIs

- 10.1063/1.1696792 may be a valid DOI for title: On the theory of electron-transfer reactions. VI. ...
- 10.1016/j.cpc.2014.04.003 may be a valid DOI for title: kmos: A lattice kinetic Monte Carlo framework
- 10.1021/acsenergylett.2c00517.s001 may be a valid DOI for title: Toward a Mechanistic Model of Solid–Electrolyte In...
- 10.26434/chemrxiv-2021-c2gp3-v2 may be a valid DOI for title: Predictive stochastic analysis of massive filter-b...
- 10.1021/jacs.3c02222.s001 may be a valid DOI for title: Chemical reaction networks explain gas evolution m...
- 10.1021/acs.nanolett.3c03568.s001 may be a valid DOI for title: Accelerating the Design of Multishell Upconverting...
- 10.1039/c4cs00205a may be a valid DOI for title: Combinatorial approaches for developing upconverti...
- 10.1021/acs.jpcc.9b00161.s001 may be a valid DOI for title: Energy transfer networks within upconverting nanop...

❌ INVALID DOIs

- None

mbarzegary commented 2 months ago

👋🏼 @Anshuman5, @lorenzo-rovigatti, @ptmerz this is the review thread for the paper. All of our communications will happen here from now on.

As a reviewer, the first step is to create a checklist for your review by entering

@editorialbot generate my checklist

as the top of a new comment in this thread.

These checklists contain the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. The first comment in this thread also contains links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews/issues/7244 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread.

We aim for reviews to be completed within about 4-6 weeks. Please feel free to ping me (@mbarzegary) if you have any questions/concerns.

editorialbot commented 2 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mbarzegary commented 2 months ago

@espottesmith this is where the review takes place. Please keep an eye out for comments here from the reviewers, as well as any issues opened by them on your software repository. I recommend you aim to respond to these as soon as possible, and you can address them straight away as they come in if you like, to ensure we do not loose track of the reviewers.

To start, can you please fix the DOIs issue raised by the editorial bot above?

Anshuman5 commented 2 months ago

Review checklist for @Anshuman5

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/BlauGroup/RNMC?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@espottesmith) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

espottesmith commented 2 months ago

All available DOIs should be present now.

lorenzo-rovigatti commented 2 months ago

Review checklist for @lorenzo-rovigatti

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/BlauGroup/RNMC?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@espottesmith) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

lorenzo-rovigatti commented 2 months ago

I have added a couple of issues to the repo. For now I can't install the software and therefore I can't test it, but I have read the paper and I found it good, apart from the following two issues:

Authorship: the list of authors is rather long compared to the people who are listed as contributors. Do all authors comply with the JOSS policies on authorship?
Software extensibility: the paper states that RNMC has been designed "to be easily extensible". Can you clarify this statement in the paper and perhaps in the documentation, which lacks a section on this matter?

ptmerz commented 1 month ago

Review checklist for @ptmerz

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/BlauGroup/RNMC?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@espottesmith) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

ptmerz commented 1 month ago

First part of my review, unfortunately I ran out of time today and will continue in the next days.

License: While the license looks like an appropriate open source license, I didn't succeed on finding it on https://opensource.org/licenses/alphabetical as requested by the checklist. If it is on the list, can you point me to it?
Contributors: As @lorenzo-rovigatti mentioned, we can only verify three of the authors (Laura Zichi, Daniel Barter, and @espottesmith) from the repo. Can you please comment on your choices for the author list?
Installation: While I could install the software without issues in one of the described ways, the other described way failed with a non-descriptive error. The underlying problem is very minor, but the resulting error can be confusing and time consuming. I opened an issue at https://github.com/BlauGroup/RNMC/issues/15.

ptmerz commented 1 month ago

And a quick note on the references in the paper: Some of the references in the text are rendered a bit weirdly. Several references either lack a space between the previous word and the parenthesis like this

Marcus theory(Marcus, 1965)

or they are rendered after the period or comma rather than before:

Butler-Volmer kinetics.(Newman & Balsara, 2021)

A quick look at recent publications (https://joss.theoj.org) confirm that they should be rendered with a space and before any period or comma.

espottesmith commented 1 month ago

Hey, I'll be addressing these questions/comments piecemeal.

First, references should be fixed.

Regarding the license, we're using the BSD-3-Clause-LBNL license. From https://spdx.org/licenses/BSD-3-Clause-LBNL.html:

This license is the same as BSD-3-Clause, but with an additional default contribution licensing clause

espottesmith commented 1 month ago

Regarding authorship:

Eric Sivonxay (@sivonxay) wrote most of the NPMC simulator for nanoparticle photophysics. He opened a PR (https://github.com/BlauGroup/RNMC/pull/3), but we ultimately decided that it'd be easier if @lzichi just took his code and did the work of unifying the various simulators.
Rohith Srinivaas Mohanakrishnan (@rohithsrinivaas) did a lot of testing and contributed to bugfixes. He was particularly helpful for testing and developing LGMC. He also worked on some code for visualization and analysis, but this code hasn't been integrated into RNMC yet.
Emory M. Chan (not sure he's on GitHub?) provided code for nanoparticle simulations to Eric, which eventually became NPMC. He also provided direct technical guidance for the development of NPMC.
Kristin A. Persson (also not on GitHub?) provided technical mentorship to several members of this team, including myself, Laura Zichi, and Rohith Srinivaas
Samuel M. Blau (@samblau) conceived of the idea of RNMC, directed its development from inception, and provided technical guidance to most of the developers. I'm sure he also did some amount of testing, code review, etc., though @danielbarter might be able to speak to that with more detail.

espottesmith commented 1 month ago

Installation issues should now be fixed, thanks to @lzichi.

To @lorenzo-rovigatti's point about extending RNMC, there's also now a page in the documentation on expanding RNMC, which is linked in the page for contributors. I can add a brief section in the manuscript as well.

espottesmith commented 1 month ago

Hey, just wanted to bump this. @ptmerz @lorenzo-rovigatti have your comments/issues been addressed? @Anshuman5 have you had a chance to read and evaluate the code/paper?

Anshuman5 commented 1 month ago

@espottesmith thanks for checking. I would be able to review this by early next week.

Anshuman5 commented 1 month ago

This work presents a highly valuable tool for the scientific community by providing a program for kMC simulation for modeling complex systems. I strongly recommend publication after addressing the following:

include current limitation of RNMC - for example, limited to simulating reactions upto two reactants and two products, in the draft or readme.md file.
in the documentation under GMC - Running GMC, include the sample output file names, logs, and directories that will be generated after running the command build/GMC --reaction_database=examples/GMC/end-to-end-test/.... Do the same for other sections - NPMC and LGMC.
RNMC is not expanded in the Summary section of the draft (line no.21).
Change the color of text 'RNMC' in logo.png. It is not clearly visible when opening github in dark mode. See image.

lorenzo-rovigatti commented 1 month ago

I have completed my checklist, as far as I'm concerned the submission can proceed. Thanks to the authors for changes made in response of my comments!

espottesmith commented 1 month ago

Thank you for your comments, @Anshuman5. I've now addressed all of them. I added a "limitations" section to the README, included the full name of RNMC in the paper draft, and made the logo gray rather than black for dark mode users.

To your point about file/directory generation: no new files/directories are generated by running GMC/NPMC/LGMC. All data is merely inserted into an existing database file. I have now clarified this in the docs.

espottesmith commented 1 month ago

@ptmerz, any further comments?

espottesmith commented 3 weeks ago

@editorialbot generate pdf

editorialbot commented 3 weeks ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

espottesmith commented 2 weeks ago

Pinging @ptmerz again.

espottesmith commented 1 week ago

Yo @mbarzegary, 2 of 3 reviewers have approved the manuscript. You can also verify that we've addressed the comments of @ptmerz. Is it possible to move this review forward?

ptmerz commented 1 week ago

Apologies, this has been a very busy time. I'll finish my review by the end of the weekend.

mbarzegary commented 1 week ago

Thank you @Anshuman5 and @lorenzo-rovigatti for completing your reviews.

No worries @ptmerz. I appreciate it if you finalize your review as soon as you can.

mbarzegary commented 1 week ago

@espottesmith No it is not. We need the approval of all 3 reviewers to move forward.

espottesmith commented 1 week ago

Understood, and very sorry @ptmerz for pestering.

ptmerz commented 1 week ago

I finished my review checklist and would strongly recommend the paper for acceptance in JOSS. The software, documentation and repo are currently in excellent shape, and the paper is well written. Thank you to the authors for addressing the issues brought up during the review!

espottesmith commented 1 week ago

Thank you so much @ptmerz!

mbarzegary commented 2 days ago

Thank you @ptmerz for completing your review.

mbarzegary commented 1 day ago

@editorialbot generate pdf

editorialbot commented 1 day ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mbarzegary commented 1 day ago

@editorialbot check references

editorialbot commented 1 day ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1063/1.1696792 is OK
- 10.1016/j.cpc.2014.04.003 is OK
- 10.1021/acsenergylett.2c00517 is OK
- 10.1039/D2DD00117A is OK
- 10.1021/jacs.3c02222 is OK
- 10.1021/acs.nanolett.3c03568 is OK
- 10.1039/C4CS00205A is OK
- 10.1021/acs.nanolett.3c01955 is OK
- 10.1021/acs.jpcc.9b00161 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Exact stochastic simulation of coupled chemical re...
- No DOI given, and none found for title: Crossing the mesoscale no-mans land via parallel k...
- No DOI given, and none found for title: Electrochemical Systems

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None

mbarzegary commented 1 day ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Double check authors and affiliations (including ORCIDs)
Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
Archive the release on Zenodo/figshare/etc and post the DOI here.
Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.
Make sure that the license listed for the archive is the same as the software license.

Editor Tasks Prior to Acceptance

[ ] Read the text of the paper and offer comments/corrections (as either a list or a pull request)
[ ] Check that the archive title, author list, version tag, and the license are correct
[ ] Set archive DOI with @editorialbot set <DOI here> as archive
[ ] Set version with @editorialbot set <version here> as version
[ ] Double check rendering of paper with @editorialbot generate pdf
[ ] Specifically check the references with @editorialbot check references and ask author(s) to update as needed
[ ] Recommend acceptance with @editorialbot recommend-accept

mbarzegary commented 1 day ago

@espottesmith given the green light of the reviewers, we will now work towards processing this for acceptance in JOSS. So please

Merge my PR with some minor edits
Work on the author's points of the final checklist I created ☝️

I can then move forward with recommending acceptance of the submission.

espottesmith commented 1 day ago

Thank you @mbarzegary!

I've merged in your PR, and in response to your checklist above:

Authors and affiliations are correct
The most recent release, which includes all changes related to this review, is https://github.com/BlauGroup/RNMC/releases/tag/v1.1.0
There's currently an issue with the RNMC Zenodo (https://zenodo.org/records/7277767), and recent releases haven't been posted and archived there. I'll work to get that sorted as soon as possible. Once I am able to post releases there, I'll also update the author list and license. The DOI is and will be https://doi.org/10.5281/zenodo.7277766

espottesmith commented 1 day ago

@editorialbot generate pdf

editorialbot commented 1 day ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

openjournals / joss-reviews

[REVIEW]: RNMC: kinetic Monte Carlo implementations for complex reaction networks #7244

Status

Reviewer instructions & questions

Checklists

Review checklist for @Anshuman5

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @lorenzo-rovigatti

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @ptmerz

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance