Create a vignette for users

[maelle]

and maybe link its pkgdown version from covrpage report-as-vignette + report-as-README.

"This package extends the information that is communicated with the potential user with a cover page for the tests in the form of summary tables of results from covr and testthat." make the potential user understand how to use the information.

I am not sure how to use it: would you tell users to stay away from a version with failing tests for instance?

There's also an aspect covrpage can't simplify, I think: you might have a failing test, and a corresponding issue in the issue tracker. In the covrpage report, one would only see the failing test, no link to that existing issue.

[yonicd]

"I am not sure how to use it: would you tell users to stay away from a version with failing tests for instance?"

personally my levels of package usage is

• travis test badge
  • pass
    • i dont really look anymore at coverage percent for a package since the distribution is more informative for me, that number can be manipulated in too many ways to look good or bad.
    • look at distribution of coverage (table 1)
    • check if there are whole groups of functionalities that were not tested.
    • if i need the line by line coverage i go to coveralls and dig in there. ( i tried adding a link to the coveralls of a build, but that was too difficult and many times people don't have it turned on, or are using private/ghe repos so their coverage isnt public)
    • look at test stats in (table 2)
      • summary of what tests contain a lot of skipped/warning, is it concentrated in only one or is it all over the tested files.
      • gives idea where to look in (table 3) for specific tests
    • look at breakdown of tests (table 3)
    • see if they make sense and seem robust
    • see if any throw warnings or are skipped, and check why (skip_travis/skip_cran)
    • use links to jump to actual problem tests/expectations to investigate further
  • fail
    • look at (table 1) for the approximate distribution of coverage to see where it stands in the development
    • find where the failed tests are located in (table 2)
    • look at the expectations from (table 3) and have a better informed decision on wether to use that package and to know what functionality to stay away from.
    • have an idea of what to expect from the developer of the package going forward what is going to be fixed.

"There's also an aspect covrpage can't simplify, I think: you might have a failing test, and a corresponding issue in the issue tracker. In the covrpage report, one would only see the failing test, no link to that existing issue."

that seems like an over-reach, covrpage is run locally or on a ci, it has no idea what is going on in issues. covrpage gives the road map to investigate problem functionality, the user needs to be able to use/search issues to see what the status is.

[maelle]

ok. the user you have in mind is a package developer too right? or at least a very informed user?

[yonicd]

no.

simple users dont have to dive all the way in to that list. they can be taught basic information on distribution of coverage and use the summary table to see if there is a problem testing wise.

i've given 2 minute explanations to end users at work and they got it right away.

in general people do want more information than just a number, and feel more secure in their use of an unknown package when they can 'see' the names of the tests that were run and see the 'PASS' next to it. it adds a sense of more control of what they are using, instead of just going off of name recognition of the repo owner.

This also will help expand people's trust in more developers and hopefully create a more diverse ecosystem of packages being tried/used.

[yonicd]

as an example of a tests dir wo a readme. I have no idea what was tested, but dirk is serious so i'll use the package. even clicking on the tests/testthat dir only gets me so far, i would need to go file by file unpack all the expectations to start to get an idea of what is/is not tested. That is wayyyy beyond what can be expected an end user to do.

covrpage gives a shortcut to that level of info to an end user, if an advanced user needs info I am guessing they already know where/how to get it (and even then covrpage will make the advanced user's life simpler)

[maelle]

check if there are whole groups of functionalities that were not tested.

so you need to know the functionality quite well to begin with? Do you first read NAMESPACE + R/ or the docs?

[maelle]

look at (table 1) for the approximate distribution of coverage to see where it stands in the development

And/or a repo status /lifecycle badge?

[yonicd]

i dont think anyone reads NAMESPACE too much. If the docs arent informative enough I may look at the R dir but that is also ussually a complex web of functions that is hard to make sense unless it is forked and dug around for a good while.

What i meant is that from the names in the functions in table 1 I can give a good guess what is and isnt tested, package functionality wise. like ggstatsplot i can tell that the helpers still havent been looked at and when i use ggcoefstat,ggcormat,histstats to keep an eye on whati get back from them.

this can also be a good spot to motivate covr_log since they may be new files and i can see their test history using that function. I think this is also what @colinfay was thinking when he suggested adding the function.

repo status and lifecycle come before tests/REAMDE.md sequentially since they are in the root README. they inform me how much a should look closely at the testthtat details table

[yonicd]

merged PR https://github.com/yonicd/covrpage/commit/c76d9efa256ec6ca7b3154427784267c01689de8