search

insightsengineering / thevalidatoR

Github Action that generates R Package Validation documentation 🏁
https://github.com/marketplace/actions/r-package-validation-report
MIT License
57 stars 5 forks source link

What to show from risk metric #36

Closed epijim closed 2 years ago

epijim commented 2 years ago

Currently we have:

image

The last R CMD check could maybe be removed - and objects exported by the package seems suss (should that be all exported functions?)

But main question is- what should we take from riskmetric and display in this report.

dgkf-roche commented 2 years ago

A few of these rows are malformed ("objects exported by package", for example). They are vectors, but only the first value in the vector is being displayed. "objects exported by package" should be a vector containing the exported namespace names.

But main question is- what should we take from riskmetric and display in this report.

To me, this table adds very little value, and I'd be in favor of removing it in a generic report. Risk assessments aren't particularly meaningful without a way of characterizing their risk. These considerations are largely organizational preference.

If we do want to embed this information, I would probably break it up into a few separate sections. Perhaps just embedding package info (package name, version, maintainer, license) and the downloads/yr. Since this action will always be run on the source code (not the CRAN release), things like OS's with R CMD check errors during CRAN builds will always be NA.

As well, instead of filtering out coverage and r cmd check from the metrics post-hoc, we should define a list of metrics that we are interested in to calculate so that we aren't duplicating heavy coverage and check operations.

epijim commented 2 years ago

Ok - for now I kept it super light and just left in this: image

I guess right now we get very little value from riskmetric - but it's something to circle back to later. And we have the open issue about what use the score could be.

dgkf-roche commented 2 years ago

Just an FYI - we are kicking off an effort to make a more data-driven score in riskmetric. We're hoping to have a poll prepped for R/Pharma to gather user trust in various packages from which we can build a more representative algorithm.

https://github.com/pharmaR/riskmetric/issues/195#issuecomment-942541161