[REVIEW]: imodels: a python package for fitting interpretable models

whedon commented 3 years ago

Submitting author: @csinva (Chandan Singh) Repository: https://github.com/csinva/imodels Version: 0.2.8 Editor: @vissarion Reviewer: @jungtaekkim, @yxoos Archive: 10.5281/zenodo.4026886

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/28d771be56981b95fbd6898068151515"><img src="https://joss.theoj.org/papers/28d771be56981b95fbd6898068151515/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/28d771be56981b95fbd6898068151515/status.svg)](https://joss.theoj.org/papers/28d771be56981b95fbd6898068151515)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@jungtaekkim & @yxoos , please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @vissarion know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @jungtaekkim

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@csinva) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @yxoos

Conflict of interest

[ ] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@csinva) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @jungtaekkim, @yxoos it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.77 s (152.6 files/s, 41400.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
HTML                            41           2108              0          18815
Python                          58           1232           1311           4778
Jupyter Notebook                 6              0           2397            408
CSS                              1             40             21            347
Markdown                         7             96              0            300
TeX                              1              9              0             99
YAML                             2              5              2             29
Bourne Shell                     2              0              1             17
-------------------------------------------------------------------------------
SUM:                           118           3490           3732          24793
-------------------------------------------------------------------------------

Statistical information for the repository '7fe9e1bd6205f9076c7a8286' was
gathered on 2021/04/19.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Avi A                            1             1              0            0.01
Chandan Singh                   83          8601           2549           62.02
Keyan Nasseri                   63          2525           1818           24.16
keyan                           23          1513            971           13.82

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Chandan Singh              4960           57.7          3.2                8.49
Keyan Nasseri              2361           93.5          2.3                8.17

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

amirhszd commented 3 years ago

Reviewer: @yxoos Feedback is provided as below:

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.
Code of Conduct
[x] I confirm that I read and will adhere to the JOSS code of conduct.
General checks
[x] Repository: Is the source code for this software available at the repository url? The source code is available on the given URL.
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license? Yes, the repository contains an MIT License.
[x] Contribution and authorship: Has the submitting author (@csinva) made major contributions to the software? Does the full list of paper authors seem appropriate and complete? Yes, the author has made the most contribution to this project.
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines. Yes, It does meet the scope.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation? Installation is seamless on Python 3.6-3.9.
[x] Functionality: Have the functional claims of the software been confirmed? The functional claims are confirmed, and the software works as expected.
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.) There are no performance claims.

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? I believe if there was more elaboration of the problem being solved, prior to providing an example, a single paragraph of few sentences, it would improve the quality of the software (probably the summary of the submitted manuscript could be used). I think if the authors started the readme with that then showed the example, as provided, followed by the installation section (highlighted as a section), it would improve the readability of the readme file.
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution. Yes, it is mentioned and handled by installation as well.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems). Yes, it is provided.
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)? Full documentation is provided for functions.
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified? Yes, there are google colab and Jupyter notebook files included.
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support Yes, this is clearly stated in the readme file.

Software paper
[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided? Verified, complete and concise.
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is? Yes, the statement of need is provided and explains the problem that needs to be solved.
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages? Yes, there is a short literature review provided.
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)? The quality of writing is coherent and easy to read.
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax? Yes the references are complete and cited properly

Find the docx. document for this review: Reviewer_yxoos_feedback.docx .

jungtaekkim commented 3 years ago

Hi @vissarion,

I have left the review in the authors' repository.

Best, Jungtaek.

csinva commented 3 years ago

Thanks to both reviewers for their time and thoughtful comments! Glad to hear you feel the package addresses a worthy problem and works smoothly 😊.

Following a suggestion from both reviewers, we made it clearer how to install the project from pip by putting it in its own section near the top of the readme.

Following another suggestion from @yxoos we also added an introductory paragraph in the readme describing a statement of need for the package -- it is now much clearer.

Thanks also to @vissarion for quick handling of the editing so far.

Best, Chandan

vissarion commented 3 years ago

Thanks @csinva for the update. A few steps before recommending acceptance.

@yxoos it seems that your concerns about the statement of need are addressed. If you are OK with this please click the related box in your review.

@jungtaekkim it seems that there is an unclicked box in your review about community guidelines. Does this raise an issue? If not please click it to have a complete review. I personally found these guidelines (referenced from readme) enough.

jungtaekkim commented 3 years ago

@vissarion I missed checking the boxes. I saw those guideline.

jungtaekkim commented 3 years ago

I updated all the checkboxes.

amirhszd commented 3 years ago

Done, great job and thanks!

vissarion commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

vissarion commented 3 years ago

@whedon check references

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s10994-015-5528-6 is OK
- 10.1214/15-aoas848 is OK
- 10.1023/A:1022631118932 is OK
- 10.1038/s42256-019-0048-x is OK
- 10.1073/pnas.1900654116 is OK
- 10.1214/07-aoas148 is OK

MISSING DOIs

- 10.1201/9780367816377-16 may be a valid DOI for title: Interpretable machine learning

INVALID DOIs

- None

vissarion commented 3 years ago

@csinva it seems that we are close to acceptance.

Please make a tagged release and archive, and report the version number and archive DOI here.

Also whedon is complaining about a missing DOI, the proposed DOI is from a different book and I cannot find a DOI for the book you refer to so I think it is OK.

A small comment on the readability of your paper is that you can use the same single sign to indicate equal contribution for both first two authors e.g. mark both with *.

csinva commented 3 years ago

@whedon generate pdf

csinva commented 3 years ago

Thanks @vissarion!

Made the tagged release and archive on zenodo. The version is 0.2.8 and the doi is 10.5281/zenodo.4026886.

Indeed, I cannot find a doi for that reference either.

I have been unable to get the JOSS template to use the same character for equal contribution - it seems this is a known issue.

Please let me know if there's any other way I can help!

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

vissarion commented 3 years ago

@whedon set 10.5281/zenodo.4026886 as archive

whedon commented 3 years ago

OK. 10.5281/zenodo.4026886 is the archive.

vissarion commented 3 years ago

@whedon set 0.2.8 as version

whedon commented 3 years ago

OK. 0.2.8 is the version.

vissarion commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

vissarion commented 3 years ago

@csinva thanks for the feedback!

The list of authors in the paper is not the same as the list in 10.5281/zenodo.4026886

Could you please fix this?

csinva commented 3 years ago

Just fixed, thanks!

vissarion commented 3 years ago

@whedon accept

whedon commented 3 years ago

Attempting dry run of processing paper acceptance...

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s10994-015-5528-6 is OK
- 10.1214/15-aoas848 is OK
- 10.1023/A:1022631118932 is OK
- 10.1038/s42256-019-0048-x is OK
- 10.1073/pnas.1900654116 is OK
- 10.1214/07-aoas148 is OK

MISSING DOIs

- 10.1201/9780367816377-16 may be a valid DOI for title: Interpretable machine learning

INVALID DOIs

- None

whedon commented 3 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/2265

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/2265, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

whedon commented 3 years ago

:wave: @yxoos , please update us on how your review is going (this is an automated reminder).

amirhszd commented 3 years ago

@whedon accept deposit=true

whedon commented 3 years ago

I'm sorry @yxoos, I'm afraid I can't do that. That's something only editor-in-chiefs are allowed to do.

amirhszd commented 3 years ago

@whedon accept

whedon commented 3 years ago

I'm sorry @yxoos, I'm afraid I can't do that. That's something only editors are allowed to do.

amirhszd commented 3 years ago

Can someone remind me what I need to do at this stage?

kyleniemeyer commented 3 years ago

Hi @yxoos, it looks like you have checked off your reviewer checklist and provided comments - so you are done!

I see in your uploaded document of comments that you did leave a comment on the paper's statement of need, which I am copying here for @csinva and @vissarion's sake:

I believe if there was more elaboration of the problem being solved, prior to providing an example, a single paragraph of few sentences, it would improve the quality of the software (probably the summary of the submitted manuscript could be used).

I think if the authors started the readme with that then showed the example, as provided, followed by the installation section (highlighted as a section), it would improve the readability of the readme file.

csinva commented 3 years ago

I believe we did update the statement of need (in this reply and the associated commit) and @yxoos found it satisfactory (based on this reply). Looks like the review is waiting on one of the EICs now?

vissarion commented 3 years ago

I also run a whedon accept. It seems everything is OK here.

On Mon, May 3, 2021, 20:27 Chandan Singh @.***> wrote:

I believe we did update the statement of need (in this reply https://github.com/openjournals/joss-reviews/issues/3192#issuecomment-825362740 and the associated commit) and @yxoos https://github.com/yxoos found it satisfactory (based on this reply https://github.com/openjournals/joss-reviews/issues/3192#issuecomment-826869370 ).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/3192#issuecomment-831410585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA35UTXHCUEH543TL2L6TODTL3MINANCNFSM43FWPD5Q .

kthyng commented 3 years ago

Ok everything looks good! We can move forward.

kthyng commented 3 years ago

@whedon accept deposit=true

whedon commented 3 years ago

Doing it live! Attempting automated processing of paper acceptance...

whedon commented 3 years ago

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

whedon commented 3 years ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/2278
Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.03192
If everything looks good, then close this review issue.
Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

kthyng commented 3 years ago

Congrats on your new publication @csinva!! Thanks to editor @vissarion and reviewers @jungtaekkim and @yxoos for your expertise, time, and hard work!

whedon commented 3 years ago

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.03192/status.svg)](https://doi.org/10.21105/joss.03192)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.03192">
  <img src="https://joss.theoj.org/papers/10.21105/joss.03192/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.03192/status.svg
   :target: https://doi.org/10.21105/joss.03192

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

openjournals / joss-reviews