ropensci / statistical-software-review-book

Guide for development and peer-review of statistical software
https://stats-devguide.ropensci.org
42 stars 11 forks source link

workflow to check formatting of standards #55

Closed mpadge closed 2 years ago

mpadge commented 2 years ago

54 from @helske is an example of the kind of mistake that can easily arise in the ongoing workflow of developing this book which relies on exactly matching hand-written standards numbers. Need to set up a GitHub workflow that injects all standards into a package, and then runs the whole srr checking workflow on them, to ensure that no glitches like that arise. Ping @noamross, to keep you aware of this helpful development.

mpadge commented 2 years ago

This is really needed - initial trial run highlights a number of issues already:

z <- system.file ("extdata", "pkgstats_9.9.tar.gz", package = "pkgstats")
path <- pkgstats::extract_tarball (z)
library (srr)
f <- file.path (path, "R", "srr-stats.R")
s <- srr_stats_roxygen (category = "bayesian", filename = f)
#> ✔ Downloaded general standards
#> ✔ Downloaded bayesian standards
#> ℹ Roxygen2-formatted standards written to [srr-stats.R]
any (grepl ("BS[0-9]", readLines (f))) # Are Bayesian stats in file?
#> [1] TRUE

r <- srr_report (path, view = FALSE)
#> ℹ Loading pkgstats
#>   ─  DONE (pkgstats)
#> 
#> ✔ Downloaded general standards
#> processing file: file62525e9b8c2.Rmd
#> output file: file62525e9b8c2.knit.md
#> 
#> Output created: /tmp/RtmprCkvzs/file62525e9b8c2.html

# Are Bayesian standards in report?
any (grepl ("BS[0-9]", r)) # Nope!!
#> [1] FALSE
r [grep ("Missing Standards", r):length (r)]
#> [1] "## Missing Standards"                                                                                                                                                                                  
#> [2] ""                                                                                                                                                                                                      
#> [3] "The following standards are missing:"                                                                                                                                                                  
#> [4] ""                                                                                                                                                                                                      
#> [5] "General standards:"                                                                                                                                                                                    
#> [6] ""                                                                                                                                                                                                      
#> [7] "G1.4a, G2.0a, G2.1a, G2.3a, G2.3b, G2.4a, G2.4b, G2.4c, G2.4d, G2.4e, G2.14a, G2.14b, G2.14c, G3.1a, G5.2a, G5.2b, G5.4a, G5.4b, G5.4c, G5.6a, G5.6b, G5.8a, G5.8b, G5.8c, G5.8d, G5.9a, G5.9b, G5.11a"
#> [8] ""

grep ("G1\\.4a", r) # But that one is not missing!
#> [1] 16 89

Created on 2021-11-22 by the reprex package (v2.0.1.9000)

mpadge commented 2 years ago

Fixed by adding a test-report.R file for ropensci-review-tools/srr#21.