search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[REVIEW]: modelDown: automated website generator with interpretable documentation for predictive machine learning models #1444

Closed whedon closed 5 years ago

whedon commented 5 years ago

Submitting author: @matiszak (Mateusz Urbański) Repository: https://github.com/MI2DataLab/modelDown Version: v1.0.1 Editor: @alexhanna Reviewer: @terrytangyuan, @pdwaggoner Archive: 10.5281/zenodo.3247303

Status

status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/d8c6609e293913021f03e2bb0ef759a1"><img src="http://joss.theoj.org/papers/d8c6609e293913021f03e2bb0ef759a1/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/d8c6609e293913021f03e2bb0ef759a1/status.svg)](http://joss.theoj.org/papers/d8c6609e293913021f03e2bb0ef759a1)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@terrytangyuan & @pdwaggoner, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @alexhanna know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @terrytangyuan

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @pdwaggoner

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 5 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @terrytangyuan, it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands
whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

pdwaggoner commented 5 years ago

@whedon check references

whedon commented 5 years ago 
Attempting to check references...
whedon commented 5 years ago 

OK DOIs

- 10.18637/jss.v082.i11 is OK

MISSING DOIs

- None

INVALID DOIs

- None
pdwaggoner commented 5 years ago

Hi - Sorry for the delay. Overall, this looks really good! Here are a few comments for things I think should be addressed/fixed:

  1. The license on the main repo (Apache 2.0) doesn't match the license at the pkgdown "documentation" site (GPL-3 -- https://mi2datalab.github.io/modelDown/). Both are acceptable by OSI, but I would clarify and fix.

  2. Installation instructions are included in the vignette, but I would like to see something a little more clear and up front on the repo README landing (e.g., Installation for Dev Version: xxx // Installation for Stable Version: xxx). Something simple and straightforward for the user would be a useful place to start interaction with the package.

  3. Typo on page 1 in the paragraph starting with "The modelDown package is designed..." in the sentence, "In THE case of pkgdown... while in THE case of modelDown..."

  4. Typo on page 2, in the paragraph starting with "The 'Model performance'..." -- the word "lab" should be "tab". There are many other small grammatical errors/typos throughout. I would recommend a closer second (or third) read-through of the paper to root these out and correct them. Should be a simple fix.

  5. I don't think the first sentence of the Summary section (which is slightly grammatically incorrect) is too useful.

  6. I am not sure if the authors were intending to include a sample screenshot or figure in the two places with hyphens (e.g., "-Example model performance tab."), but if not, I would highly recommend including a screenshot to give the user/reader an idea of what the output looks like. If so, then perhaps the authors forgot to include this in the manuscript. I would include some demo figures either way.

  7. An example/demo of the code would be a helpful addition to the paper. Perhaps just before the explanation of all of the tabs produced in the output. It would be nice just to see the code/package in action within the JOSS paper, even though examples are included in the package vignette.

  8. I would also include a note on how people can contribute (if you want them to), via issue tickets, PRs, or whatever this might look like.

Other than these things, the paper and software all looks great! The authors did a good job here and I think the package will be widely accepted and used. Great work!

Matiszak commented 5 years ago

Thank you very much for your review and sorry for the delay. I hope I took care of all the points, your input was very helpful. Could you see if there is anything else worth changing ?

Matiszak commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

pdwaggoner commented 5 years ago

This all looks really great to me! Thanks for making the changes. I would say its good to go from my perspective. @alexhanna back over to you. Thanks!

terrytangyuan commented 5 years ago

Just went over the paper and the checklist. The paper looks great to me. Great job @Matiszak!

alexhanna commented 5 years ago

Thank you for your reviews, @pdwaggoner and @terrytangyuan! And for making the edits, @Matiszak.

alexhanna commented 5 years ago

@whedon accept

whedon commented 5 years ago

No archive DOI set. Exiting...

alexhanna commented 5 years ago

@Matiszak do you have an archive DOI for the repository? We need that for acceptance.

arfon commented 5 years ago

@Matiszak do you have an archive DOI for the repository? We need that for acceptance.

Also, @Matiszak - please make sure that you make new release of this software that includes the changes that have resulted from this review. Then, please make an archive of the software in Zenodo/figshare/other service and update this thread with the DOI of the archive. For the Zenodo/figshare archive, please make sure that:

Matiszak commented 5 years ago

The 1.0.1 version including the changes that have resulted from this review has been archived at 10.5281/zenodo.3247303 Also it set as official release on https://github.com/MI2DataLab/modelDown and also published to CRAN: https://cran.r-project.org/web/packages/modelDown/index.html

Thank you very much for your work, please let me know if there is anything else to do.

alexhanna commented 5 years ago

@whedon set v1.0.1 as version

whedon commented 5 years ago

OK. v1.0.1 is the version.

alexhanna commented 5 years ago

@whedon set 10.5281/zenodo.3247303 as archive

whedon commented 5 years ago

OK. 10.5281/zenodo.3247303 is the archive.

alexhanna commented 5 years ago

@whedon accept

whedon commented 5 years ago 
Attempting dry run of processing paper acceptance...
whedon commented 5 years ago 

OK DOIs

- 10.18637/jss.v082.i11 is OK

MISSING DOIs

- None

INVALID DOIs

- None
whedon commented 5 years ago

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/781

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/781, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true
alexhanna commented 5 years ago

@openjournals/joss-eics this looks ready for acceptance.

danielskatz commented 5 years ago

I've suggested a bunch of text changes for the paper in https://github.com/MI2DataLab/modelDown/pull/72

Matiszak commented 5 years ago

@danielskatz Thank you, the PR is merged.

danielskatz commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

danielskatz commented 5 years ago

@whedon accept

whedon commented 5 years ago 
Attempting dry run of processing paper acceptance...
whedon commented 5 years ago 

OK DOIs

- 10.18637/jss.v082.i11 is OK

MISSING DOIs

- None

INVALID DOIs

- None
whedon commented 5 years ago

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/794

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/794, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true
danielskatz commented 5 years ago

👋 @Matiszak - please also merge https://github.com/MI2DataLab/modelDown/pull/74, which has fixes for the cases of words in the bibtex. Then ping @openjournals/joss-eics for the final acceptance

Matiszak commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

Matiszak commented 5 years ago

@openjournals/joss-eics Merged :+1:

danielskatz commented 5 years ago

@whedon accept

whedon commented 5 years ago 
Attempting dry run of processing paper acceptance...
whedon commented 5 years ago

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/797

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/797, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true
whedon commented 5 years ago 

OK DOIs

- 10.18637/jss.v082.i11 is OK

MISSING DOIs

- None

INVALID DOIs

- None
danielskatz commented 5 years ago

@whedon accept deposit=true

whedon commented 5 years ago 
Doing it live! Attempting automated processing of paper acceptance...
whedon commented 5 years ago

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

whedon commented 5 years ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/798
  2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01444
  3. If everything looks good, then close this review issue.

  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

    Any issues? notify your editorial technical team...