Wrap covr::package_coverage(type = c("examples", "vignettes") to highlight exported but undocumented functions

[rmflight]

A big part of good open science is having good documentation. However, very often we don't think to examine what functionality actually gets covered in the documentation of our code, either in R package vignettes or function examples. I didn't even know covr provides code coverage reports for examples and vignettes, as the default is "tests".

But examples and vignettes don't necessarily need to cover everything or execute every line of code in a package, but should probably execute a good percentage of the exported code in the package, and if they don't, then the developer should consider whether they should be exporting it, or if they need to include it in the vignette, or develop another tutorial completely.

[jimhester]

This is a good idea, Bioconductor does something similar, requiring exported functions to have runnable examples.

Here is an simple implementation, probably something similar could be added to covr.

library(readr)
library(magrittr)
library(covr)

per_function <- function(df) {
  tapply(df$value, df$functions, FUN = function(x) (sum(x > 0) / length(x)) * 100)
}

exported_coverage <- function(..., coverage = package_coverage(...)) {
  # Assumes the package is installed
  exports <- getNamespaceExports(attr(coverage, "package")$package)

  coverage %>%
    as.data.frame() %>%
    subset(functions %in% exports) %>%
    per_function() ->
  percents

  percents[] <- vapply(percents, covr:::format_percentage, character(1))

  cat(paste0(crayon::bold(names(percents)), ": ", percents), sep = "\n")

  invisible(percents)
}

exported_coverage("~/p/tidyr", type = "example")
#> complete: 75.00%
#> complete_: 100.00%
#> crossing: 0.00%
#> crossing_: 80.00%
#> expand: 100.00%
#> expand_: 100.00%
#> extract: 100.00%
#> extract_: 100.00%
#> extract_numeric: 100.00%
#> fill: 100.00%
#> fill_: 100.00%
#> full_seq: 100.00%
#> gather: 83.33%
#> gather_: 100.00%
#> nest: 100.00%
#> nest_: 100.00%
#> nesting: 100.00%
#> replace_na: 100.00%
#> separate: 100.00%
#> separate_: 100.00%
#> separate_rows: 100.00%
#> separate_rows_: 100.00%
#> spread: 100.00%
#> spread_: 60.00%
#> unite: 100.00%
#> unite_: 100.00%
#> unnest: 100.00%
#> unnest_: 100.00%
exported_coverage("~/p/tidyr", type = "vignette")
#> complete: 0.00%
#> complete_: 0.00%
#> crossing: 0.00%
#> crossing_: 0.00%
#> expand: 0.00%
#> expand_: 0.00%
#> extract: 0.00%
#> extract_: 0.00%
#> extract_numeric: 100.00%
#> fill: 0.00%
#> fill_: 0.00%
#> full_seq: 0.00%
#> gather: 83.33%
#> gather_: 100.00%
#> nest: 0.00%
#> nest_: 0.00%
#> nesting: 0.00%
#> replace_na: 0.00%
#> separate: 100.00%
#> separate_: 100.00%
#> separate_rows: 0.00%
#> separate_rows_: 0.00%
#> spread: 100.00%
#> spread_: 60.00%
#> unite: 0.00%
#> unite_: 0.00%
#> unnest: 0.00%
#> unnest_: 0.00%

Also @rmflight as you seem interested in code coverage perhaps you would like to join the Code Coverage Working Group?

[rmflight]

I'm glad you think it is a good idea @jimhester. I'm likely to work on a full pull request for this type of thing if it doesn't get picked up for the runconf.

I am interested in code coverage, but I'm also interested in lots of other things, and my research slate keeps me extremely busy. I appreciate the offer, but I don't have time in my life right now to commit to being a member of a working group. I will keep it in mind, however.