openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
694 stars 36 forks source link

[REVIEW]: FIGARO: hierarchical non-parametric inference for population studies #6589

Closed editorialbot closed 1 month ago

editorialbot commented 3 months ago

Submitting author: !--author-handle-->@sterinaldi<!--end-author-handle-- (Stefano Rinaldi) Repository: https://github.com/sterinaldi/FIGARO Branch with paper.md (empty if default branch): joss-paper Version: v1.6.7 Editor: !--editor-->@dfm<!--end-editor-- Reviewers: @dgerosa, @Uddiptaatwork Archive: 10.5281/zenodo.11302325

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/d5e757291a0287f2fef1466c7eacdff5"><img src="https://joss.theoj.org/papers/d5e757291a0287f2fef1466c7eacdff5/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/d5e757291a0287f2fef1466c7eacdff5/status.svg)](https://joss.theoj.org/papers/d5e757291a0287f2fef1466c7eacdff5)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dgerosa & @Uddiptaatwork, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @dfm know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Checklists

📝 Checklist for @dgerosa

📝 Checklist for @Uddiptaatwork

editorialbot commented 3 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 3 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.05 s (1238.1 files/s, 154076.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          21            626           1525           3152
Markdown                         6             80              0            390
TeX                              1             10              0            179
Jupyter Notebook                 1              0            675            148
reStructuredText                27            304            184             61
TOML                             1              6              0             58
INI                              1              1              0             29
YAML                             2              5              6             28
-------------------------------------------------------------------------------
SUM:                            60           1032           2390           4045
-------------------------------------------------------------------------------

Commit count by author:

   998  Stefano Rinaldi
   169  sterinaldi
     6  Walter Del Pozzo
     2  Riccardo Buscicchio
     1  Vera Eris Del Favero
     1  d-sanfratello
editorialbot commented 3 months ago

Paper file info:

📄 Wordcount for paper.md is 959

✅ The paper includes a Statement of need section

editorialbot commented 3 months ago

License info:

✅ License found: MIT License (Valid open source OSI approved license)

editorialbot commented 3 months ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/mnras/stab3224 is OK
- 10.1093/mnrasl/slac101 is OK
- 10.48550/arXiv.2308.12182 is OK
- 10.1140/epjc/s10052-023-12078-6 is OK
- 10.1093/mnras/stad2768 is OK
- 10.1103/PhysRevD.108.123039 is OK
- 10.1140/epjc/s10052-023-11754-x is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- No DOI given, and none found for title: Evidence for the evolution of black hole mass func...
- 10.2307/2291069 may be a valid DOI for title: Bayesian Density Estimation and Inference Using Mi...
- No DOI given, and none found for title: Scikit-learn: Machine Learning in Python

INVALID DOIs

- None
dfm commented 3 months ago

@dgerosa, @Uddiptaatwork — This is the review thread for the paper. All of our correspondence will happen here from now on. Thanks again for agreeing to participate!

👉 Please read the "Reviewer instructions & questions" in the first comment above, and generate your checklists by commenting @editorialbot generate my checklist on this issue ASAP. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#6589 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please try to make a start ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule. Please get your review started as soon as possible!

editorialbot commented 3 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

dgerosa commented 2 months ago

Review checklist for @dgerosa

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

dgerosa commented 2 months ago

Hi @sterinaldi, nice work. Here are my comments following the checklist above:

dgerosa commented 2 months ago

I also have a comment/question for @dfm, which is about the scope of JOSS. The authors already published a paper in the scientific literature that presents their FIGARO code:

Monthly Notices of the Royal Astronomical Society: Letters, Volume 517, Issue 1, November 2022, Pages L5–L10, https://doi.org/10.1093/mnrasl/slac101

This JOSS paper also presents the code and their other paper provides more details. Should there be a JOSS paper for this code? Should this submission be considered a duplicate?

dfm commented 2 months ago

@dgerosa — Thanks for checking in! I normally try to check this before the review starts, but sometimes things slip through. When submitting, @sterinaldi commented "This code and documentation has not been submitted, nor we are planning to submit it, to another peer-reviewed journal.", but I agree that there does seem to be significant overlap with that MNRAS paper.

Our policy on co-publication is described here. The idea is that it is ok to publish a JOSS paper when the methods or applications have been published elsewhere, but it's not ok to co-publish primarily about the software in multiple venues.

@sterinaldi — Perhaps you could provide your perspective on the relationship between this submission and the previous publication. Thanks!

sterinaldi commented 2 months ago

Hi @dfm – It is indeed true that we already presented a part of FIGARO in Rinaldi & Del Pozzo (2022) [RDP, hereafter] as mentioned by @dgerosa. From my perspective, however, it satisfies JOSS policy on co-publication for the following reasons:

I hope this answers your questions, but I'm happy to comment more on this if you feel that it's needed!

Uddiptaatwork commented 2 months ago

Review checklist for @Uddiptaatwork

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

dfm commented 2 months ago

@sterinaldi — Thanks for your response above and I'm so sorry about my delayed reply. This does sound to me like this submission is sufficiently different from the MNRAS publication to be consistent with the JOSS requirements. With this in mind, I think we should continue with the review. Thanks again to @dgerosa for checking in on this point!

sterinaldi commented 2 months ago

@dfm – Thanks! I'll address the comments by @dgerosa in the next few days.

sterinaldi commented 1 month ago

Hi @dgerosa, sorry for my late reply. Thanks for your review, below you find my reply and changes:

I hope this addresses your comments!

dgerosa commented 1 month ago

Thanks @sterinaldi. Green light from me.

dfm commented 1 month ago

Thanks @dgerosa!

@Uddiptaatwork — A reminder to keep this on your radar. Please revisit your review ASAP!

Uddiptaatwork commented 1 month ago

Dear @sterinaldi @dfm apologies for the delay, we experienced a disruptive few weeks at the institute due to ongoing geopolitical issues. I have just finished reviewing the paper draft and code documentation also keeping in mind the points raised by @dgerosa in this context. I am currently running tests and reviewing installation specifics on MacOS(intel) and Linux and will confirm very shortly on the status.

Uddiptaatwork commented 1 month ago

All the warnings are however DeprecationWarnings and can be ignored for the time being. However, specific version updates to environment modules can cause future errors in executing certain parts of the code. So, this is just a request to the authors to look into this just to ensure future compatibility.

In any case, I recommend this article for publication and congratulate the authors for a great piece of work.

sterinaldi commented 1 month ago

Hi @Uddiptaatwork, thanks!

Concerning the warnings, I'm not really sure about what's going on there, to be honest. I was aware of these warnings (MacOS, M1), but they are relative to a documentation line (the first two, the fact that I wrote \int in the documentation causes this warning) and a plot label (the math environment space in LaTeX). The code runs smoothly without warnings outside testing, so my best guess for this is something related to the testing suite. I'll put it on my to do list for future investigation anyway.

Edit: I just checked, it's an issue with backslashes. I simply converted individual backslashes to double backslashes (https://stackoverflow.com/questions/52335970/how-to-fix-syntaxwarning-invalid-escape-sequence-in-python).

Uddiptaatwork commented 1 month ago

Hi @Uddiptaatwork, thanks!

Concerning the warnings, I'm not really sure about what's going on there, to be honest. I was aware of these warnings (MacOS, M1), but they are relative to a documentation line (the first two, the fact that I wrote \int in the documentation causes this warning) and a plot label (the math environment space in LaTeX). The code runs smoothly without warnings outside testing, so my best guess for this is something related to the testing suite. I'll put it on my to do list for future investigation anyway.

Edit: I just checked, it's an issue with backslashes. I simply converted individual backslashes to double backslashes (https://stackoverflow.com/questions/52335970/how-to-fix-syntaxwarning-invalid-escape-sequence-in-python).

Perfect, thanks for looking into this! @dfm I confirm my recommendation to publish this article.

dfm commented 1 month ago

@editorialbot generate pdf

dfm commented 1 month ago

@editorialbot check references

editorialbot commented 1 month ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/mnras/stab3224 is OK
- 10.1093/mnrasl/slac101 is OK
- 10.1051/0004-6361/202348161 is OK
- 10.48550/arXiv.2308.12182 is OK
- 10.1140/epjc/s10052-023-12078-6 is OK
- 10.1093/mnras/stad2768 is OK
- 10.1103/PhysRevD.108.123039 is OK
- 10.1140/epjc/s10052-023-11754-x is OK
- 10.1080/01621459.1995.10476550 is OK
- 10.48550/arXiv.1201.0490 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.1103/PhysRevX.13.011048 is OK
- 10.1088/1361-6382/ac0b54 is OK
- 10.3847/1538-4357/acb5ed is OK
- 10.1093/mnras/stad2215 is OK
- 10.1103/PhysRevX.14.021005 is OK
- 10.1007/978-0-387-30164-8_219 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 1 month ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

dfm commented 1 month ago

@dgerosa, @Uddiptaatwork — Thanks for your thorough and constructive reviews!!

@sterinaldi — I've opened a small PR with some minor edits to the manuscript, please take a look and merge or let me know what you think.

Once you've done that:

  1. Take one last read through the manuscript to make sure that you're happy with it (it's harder to make changes later!), especially the author names and affiliations. I've taken a pass and it looks good to me!
  2. Increment the version number of the software and report that version number back here.
  3. Create an archived release of that version of the software (using Zenodo or something similar). Please make sure that the metadata (title and author list) exactly match the paper. Then report the DOI of the release back to this thread.
sterinaldi commented 1 month ago

Hi @dfm – I merged your PR. The updated version number of the code is v1.6.7 and the DOI is 10.5281/zenodo.11302325

dfm commented 1 month ago

@editorialbot set v1.6.7 as version

editorialbot commented 1 month ago

Done! version is now v1.6.7

dfm commented 1 month ago

@editorialbot set 10.5281/zenodo.11302325 as archive

editorialbot commented 1 month ago

Done! archive is now 10.5281/zenodo.11302325

dfm commented 1 month ago

@editorialbot check references

dfm commented 1 month ago

@editorialbot generate pdf

editorialbot commented 1 month ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/mnras/stab3224 is OK
- 10.1093/mnrasl/slac101 is OK
- 10.1051/0004-6361/202348161 is OK
- 10.48550/arXiv.2308.12182 is OK
- 10.1140/epjc/s10052-023-12078-6 is OK
- 10.1093/mnras/stad2768 is OK
- 10.1103/PhysRevD.108.123039 is OK
- 10.1140/epjc/s10052-023-11754-x is OK
- 10.1080/01621459.1995.10476550 is OK
- 10.48550/arXiv.1201.0490 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.1103/PhysRevX.13.011048 is OK
- 10.1088/1361-6382/ac0b54 is OK
- 10.3847/1538-4357/acb5ed is OK
- 10.1093/mnras/stad2215 is OK
- 10.1103/PhysRevX.14.021005 is OK
- 10.1007/978-0-387-30164-8_219 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 1 month ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

dfm commented 1 month ago

@editorialbot recommend-accept

editorialbot commented 1 month ago
Attempting dry run of processing paper acceptance...
editorialbot commented 1 month ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/mnras/stab3224 is OK
- 10.1093/mnrasl/slac101 is OK
- 10.1051/0004-6361/202348161 is OK
- 10.48550/arXiv.2308.12182 is OK
- 10.1140/epjc/s10052-023-12078-6 is OK
- 10.1093/mnras/stad2768 is OK
- 10.1103/PhysRevD.108.123039 is OK
- 10.1140/epjc/s10052-023-11754-x is OK
- 10.1080/01621459.1995.10476550 is OK
- 10.48550/arXiv.1201.0490 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.1103/PhysRevX.13.011048 is OK
- 10.1088/1361-6382/ac0b54 is OK
- 10.3847/1538-4357/acb5ed is OK
- 10.1093/mnras/stad2215 is OK
- 10.1103/PhysRevX.14.021005 is OK
- 10.1007/978-0-387-30164-8_219 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 1 month ago

:wave: @openjournals/aass-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/5388, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

dfm commented 1 month ago

@editorialbot accept

editorialbot commented 1 month ago
Doing it live! Attempting automated processing of paper acceptance...
editorialbot commented 1 month ago

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

``` cff-version: "1.2.0" authors: - family-names: Rinaldi given-names: Stefano orcid: "https://orcid.org/0000-0001-5799-4155" - family-names: Del Pozzo given-names: Walter orcid: "https://orcid.org/0000-0003-3978-2030" doi: 10.5281/zenodo.11302325 message: If you use this software, please cite our article in the Journal of Open Source Software. preferred-citation: authors: - family-names: Rinaldi given-names: Stefano orcid: "https://orcid.org/0000-0001-5799-4155" - family-names: Del Pozzo given-names: Walter orcid: "https://orcid.org/0000-0003-3978-2030" date-published: 2024-05-25 doi: 10.21105/joss.06589 issn: 2475-9066 issue: 97 journal: Journal of Open Source Software publisher: name: Open Journals start: 6589 title: "FIGARO: hierarchical non-parametric inference for population studies" type: article url: "https://joss.theoj.org/papers/10.21105/joss.06589" volume: 9 title: "FIGARO: hierarchical non-parametric inference for population studies" ```

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

editorialbot commented 1 month ago

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

editorialbot commented 1 month ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/5389
  2. Wait five minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.06589
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

dfm commented 1 month ago

@dgerosa, @Uddiptaatwork — Many thanks for your reviews here! JOSS relies upon the volunteer effort of people like you and we simply wouldn't be able to do this without you!!

@sterinaldi — Your paper is now accepted and published in JOSS! :zap::rocket::boom:

editorialbot commented 1 month ago

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.06589/status.svg)](https://doi.org/10.21105/joss.06589)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.06589">
  <img src="https://joss.theoj.org/papers/10.21105/joss.06589/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.06589/status.svg
   :target: https://doi.org/10.21105/joss.06589

This is how it will look in your documentation:

DOI

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following: