editorialbot commented 7 months ago

Submitting author: !--author-handle-->@jasonkaye@kyleniemeyer<!--end-editor-- Reviewers: @jarvist, @stevenrbrandt Archive: 10.5281/zenodo.13177024

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/8fd2af732c1adedb99b271dfc91d7e1e"><img src="https://joss.theoj.org/papers/8fd2af732c1adedb99b271dfc91d7e1e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/8fd2af732c1adedb99b271dfc91d7e1e/status.svg)](https://joss.theoj.org/papers/8fd2af732c1adedb99b271dfc91d7e1e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@jarvist and @stevenrbrandt, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kyleniemeyer know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @jarvist

📝 Checklist for @stevenrbrandt

editorialbot commented 7 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 7 months ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.06 s (1060.2 files/s, 143707.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                             17            728            872           1735
C/C++ Header                     7            346           1069            822
CMake                           15            116            205            600
reStructuredText                 8            192            122            463
CSS                              2             80             20            354
TeX                              1             27              0            286
XML                              1              6              8            226
Markdown                         5             53              0            161
YAML                             4             12              6            130
MATLAB                           1             14             11             66
HTML                             2              6              0             60
JavaScript                       2              0             13              2
-------------------------------------------------------------------------------
SUM:                            65           1580           2326           4905
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 7 months ago

Wordcount for paper.md is 895

editorialbot commented 7 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1103/PhysRevB.98.035104 is OK
- 10.1103/PhysRevB.105.235115 is OK
- 10.1007/s10444-023-10067-7 is OK
- 10.1103/PhysRevB.107.245123 is OK
- 10.1103/PhysRevB.96.035147 is OK
- 10.1103/PhysRevB.84.075145 is OK
- 10.1063/5.0003145 is OK
- 10.1103/PhysRevB.98.075127 is OK
- 10.1103/PhysRev.139.A796 is OK
- 10.3389/fchem.2019.00377 is OK
- 10.1103/PhysRevB.106.L220502 is OK

MISSING DOIs

- 10.1103/physrevresearch.6.013099 may be a valid DOI for title: Precursory Cooper Flow in Ultralow-Temperature Superconductors
- 10.1103/physrevb.108.184501 may be a valid DOI for title: A comparative study of the superconductivity in the Holstein and optical Su-Schrieffer-Heeger models

INVALID DOIs

- https://doi.org/10.1016/j.cpc.2022.108458 is INVALID because of 'https://doi.org/' prefix
- http://dx.doi.org/10.1016/j.cpc.2015.04.023 is INVALID because of 'https://doi.org/' prefix
- https://doi.org/10.1016/j.softx.2022.101266 is INVALID because of 'https://doi.org/' prefix

editorialbot commented 7 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jasonkaye commented 5 months ago

@editorialbot generate preprint

editorialbot commented 5 months ago

:page_facing_up: Preprint file created: Find it here in the Artifacts list :page_facing_up:

jasonkaye commented 5 months ago

@editorialbot generate pdf

editorialbot commented 5 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jasonkaye commented 5 months ago

@editorialbot generate preprint

editorialbot commented 5 months ago

:page_facing_up: Preprint file created: Find it here in the Artifacts list :page_facing_up:

jarvist commented 4 months ago

I'm sorry that this review is taking so long. @Neutrino155 has had to withdraw for personal reasons.

I've found it extremely difficult to find an expert reviewer - most people seem quite reticent to express an opinion given the standing of the authors in the field.

I will now take the unusual step of reviewing it myself, while continuing to search for an expert to weigh in with an external view of the physics.

jarvist commented 4 months ago

@editorialbot remove @Neutrino155 from reviewers

editorialbot commented 4 months ago

@Neutrino155 is not in the reviewers list

jarvist commented 4 months ago

@editorialbot add @jarvist to reviewers

editorialbot commented 4 months ago

@jarvist added to the reviewers list!

jarvist commented 4 months ago

Review checklist for @jarvist

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/flatironinstitute/cppdlr/?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@jasonkaye) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

kyleniemeyer commented 2 months ago

@editorialbot assign me as editor

editorialbot commented 2 months ago

Assigned! @kyleniemeyer is now the editor

kyleniemeyer commented 2 months ago

@editorialbot add @stevenrbrandt as reviewer

Steve Brandt has agreed to provide another review within the next few weeks or so. Thanks!

editorialbot commented 2 months ago

@stevenrbrandt added to the reviewers list!

stevenrbrandt commented 2 months ago

OK, I'm getting started and walking my way through the checklist. I don't see anything that obviously fits the definition "A statement of need" in the docs. To me, a statement of need should probably have a title like "why this package was created" or something of that sort.

stevenrbrandt commented 2 months ago

Also, where is "the paper" mentioned in the checklist above?

stevenrbrandt commented 2 months ago

There seems to be a single example: https://flatironinstitute.github.io/cppdlr/main/examples.html

I was able to compile and run it with ease, and it produced a number of data files. I think it would be nice to see the scripts used to create the plots on the examples page from the data.

stevenrbrandt commented 2 months ago

I am also having trouble identifying "community guidelines." All I see is that, to report issues, they authors point people to the github issue tracker.

stevenrbrandt commented 2 months ago

I should probably add that the science the authors are doing is barely comprehensible to me as it's totally outside my field.

Again, I'm not sure what the requirements are for the statement of need, but one thing I would like to see is a high-level list of scientific questions / projects this code is used for, e.g. analysis of results from the LHC or something. Maybe some text borrowed from a community outreach event.

kyleniemeyer commented 2 months ago

@editorialbot generate pdf

editorialbot commented 2 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

kyleniemeyer commented 2 months ago

@stevenrbrandt the statement of need is in the paper 👆, linked to here (also earlier in the review issue)

stevenrbrandt commented 2 months ago

Review checklist for @stevenrbrandt

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/flatironinstitute/cppdlr/?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@jasonkaye) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

kyleniemeyer commented 1 month ago

Hi @jarvist and @stevenrbrandt, just wanted to check on the status of your reviews. Thank you!

stevenrbrandt commented 1 month ago

@kyleniemeyer I'm going to try and finish today

stevenrbrandt commented 1 month ago

@jarvist @kyleniemeyer I have a few boxes that I haven't checked.

I do not see a "statement of need" in the docs. It's in the paper, though.

Automated tests look like they just test that the code builds, not that any result is repeatable. Is that sufficient? UPDATE: I found the tests. I'm checking the box.

While there are community guidelines, they just point users at the github issues link. Is that sufficient?

UPDATE: Since I don't see any particular performance claims, I checked the performance box.

Thanks.

jasonkaye commented 1 month ago

@stevenrbrandt Thanks for going through this carefully.

For the statement of need, I think that for our users, the Background page of our website should serve this purpose. However, if this is not sufficient, we could discuss what else needs to be added.
We have automated unit tests which test essentially all aspects of the code, including reproducing practical results. These can be found in the test/c++ folder, and should run with make test. For example, test/c++/imtime_ops.cpp and test/c++/imfreq_ops.cpp contain tests for many of the main user-facing functions provided by the library, with realistic examples checking that user-specified error tolerances are satisfied.

stevenrbrandt commented 1 month ago

@jasonkaye I really liked the statement of need in your paper much better. It is more of the form "here is the functionality we provide" and "here are other libraries which provide similar functionality." If you could make it a bit more like that, I think that would be great.

stevenrbrandt commented 1 month ago

@jasonkaye also, it would be nice if for "community guidelines" you provided some guidance for people who want to make a PR.

jasonkaye commented 1 month ago

@stevenrbrandt Thanks for the suggestions. We have added a paragraph on the front page of the documentation website which includes some of the points you mentioned. We have also renamed the "Issues" page to "Issues and contributions", and included a brief statement on making pull requests.

Note that this was done in the main (development) branch website, which is referenced and linked to in the github repo readme. It will be included in the stable branch website with our next release.

stevenrbrandt commented 1 month ago

I have checked all boxes on the form. My review is complete. The paper is ready to be published as far as I am concerned. :)

kyleniemeyer commented 1 month ago

Thanks @stevenrbrandt!

kyleniemeyer commented 1 month ago

@jarvist will you be able to complete your review very soon? Thank you!

jarvist commented 1 month ago

@editorialbot generate pdf

editorialbot commented 1 month ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jarvist commented 1 month ago

@stevenrbrandt Thanks for the suggestions. We have added a paragraph on the front page of the documentation website which includes some of the points you mentioned. We have also renamed the "Issues" page to "Issues and contributions", and included a brief statement on making pull requests.

Note that this was done in the main (development) branch website, which is referenced and linked to in the github repo readme. It will be included in the stable branch website with our next release.

I think this is a minor but very important correction. But I can't currently see the new text - either by following the above link, or poking around GitHub myself. Could you confirm that these edits will be included? (It may just be that my web browser cache is stale, etc.)

jarvist commented 1 month ago

OK! Other than just hanging on making sure the 'community guidelines' are updated, I think we are all good.

Less usual to see a C++ scientific code today, but this one looked quite nicely and cleanly written, using modern C++ templates approach. I thought the abstraction to the NDA sub-library was interesting and quite nice, but then I wondered whether it would make sense for it to for-ever live in this repository.

Code comments were quite light in simplistic parts, but then went into a nice level of detail in some of the more tricky bits of physics, which felt like the right approach.

Codebase is compact, but I think this work definitely makes the JOSS threshold for scholarly effort as its very hard to get the numerics of these things correct, and it's certainly something that will accelerate the development of codes which make use of the discrete Lehmann representation.

It would have been nice to see a bit more description of how these codes compare to the Python, Julia and previous C (?) libraries that do the same. The motivation for a modern C++ code is absolutely fine though. In an ideal world it would have been nice to have some more examples, but the one that you did have was very nicely documented with both the physics & then the code side by side.

jasonkaye commented 1 month ago

@jarvist Thanks for your comments. Thank you for catching the website error---this was caused by a bug related to the deployment of the docs to the website, and is now fixed. You should now be able to see the changes on the main branch website, which will eventually make it to the stable branch.

We plan to add more examples as the need arises. The example we have will already cover probably 90% of user needs. As we describe on the examples page, there are many other examples in the form of unit tests, and we give a map from features to the relevant unit tests on that page. Those unit tests themselves are also reasonably well documented, though not explained in as much detail as the main example.

kyleniemeyer commented 1 month ago

I can see the change on the docs website regarding community guidelines, which I believe was @jarvist's only blocking issue - and so the reviews are now complete for this submission!

kyleniemeyer commented 1 month ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Double check authors and affiliations (including ORCIDs)
Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
Archive the release on Zenodo/figshare/etc and post the DOI here.
Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.
Make sure that the license listed for the archive is the same as the software license.

Editor Tasks Prior to Acceptance

[x] Read the text of the paper and offer comments/corrections (as either a list or a pull request)
[x] Check that the archive title, author list, version tag, and the license are correct
[x] Set archive DOI with @editorialbot set <DOI here> as archive
[x] Set version with @editorialbot set <version here> as version
[x] Double check rendering of paper with @editorialbot generate pdf
[x] Specifically check the references with @editorialbot check references and ask author(s) to update as needed
[x] Recommend acceptance with @editorialbot recommend-accept

kyleniemeyer commented 1 month ago

@jasonkaye please review the post-review author tasks above ☝️

jasonkaye commented 1 month ago

@kyleniemeyer We have checked all of the above. We have also made very minor edits to the paper in our repo.

Version 1.2.0 10.5281/zenodo.13177024

Please let me know if you need anything else.

kyleniemeyer commented 1 month ago

@editorialbot set 1.2.0 as version

openjournals / joss-reviews

[REVIEW]: cppdlr: Imaginary time calculations using the discrete Lehmann representation #6297

Status

Reviewer instructions & questions

Checklists

Review checklist for @jarvist

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @stevenrbrandt

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance