ropensci / statistical-software-review-book

Guide for development and peer-review of statistical software
https://stats-devguide.ropensci.org
42 stars 11 forks source link

Review request template is not stats-specific #68

Closed noamross closed 2 years ago

noamross commented 2 years ago

The review request template (https://stats-devguide.ropensci.org/appendix.html#reviewrequesttemplate), is a copy of the one in the regular dev guide and doesn't provide guidance specific to how to review a statistical package. The following email is one I had to send as a follow-up after someone used the template, it can be a starting point:

I think we got a bit of a false start on as our reviewer request templates didn't have all our updated information.  We are in the midst of piloting approaches to review specific to statistical-algorithm implementing packages and thus focuses more on correctness and testing than interface.  Would you still be interested in reviewing the `XXXX` package for us?

Our guidance for reviewing packages of this sort is at https://stats-devguide.ropensci.org/pkgreview.html. While there specific standards, they are much more aligned towards statistical problems than what I think you saw before.  This package falls under the "XXXX" and "YYYY" categories so those subsets are what is applicable.  There's a template for reviewers at the end of that page.

We have put in place some automation to support this. You'll see that automated checks are output in the GitHub review thread here: https://github.com/ropensci/software-review/issues/XXXXX . Importantly, one of the outputs is a report of how the author has annotated their code to show compliance with all the standards requirements, as well as those they consider non-applicable. The goal of this is to free up the reviewer to focus on the human-judgement-required tasks of evaluating correctness, design, and documentation.

This program is still in pilot mode so we're of course eager to hear feedback on how well this works for people, and while we aim for 3 weeks turnaround, we're flexible on the review time.

Please let me know if you think you'd be able to review. If you can not, suggestions for alternate reviewers are always helpful. If I don't hear from you within a week, I will assume you are unable to review at this time.