search

ccao-data / actions

GitHub Actions for CCAO Data projects
0 stars 0 forks source link

Refactor `test-coverage` action for R packages #28

Open dfsnow opened 2 months ago

dfsnow commented 2 months ago

Our R package test coverage Actions (e.g. test-coverage.yaml) are failing due to https://github.com/r-lib/actions/issues/834. We should refactor this action into a shared composite action in this repo. We should also take this opportunity to remove the Codecov integration (since they're increasingly moving to a paid model). Let's replace it with a simple coverage badge with a check that fails if coverage is reduced.

jeancochrane commented 1 month ago

@dfsnow See some discussion here around the complexity of measuring reductions in coverage: https://community.codecov.com/t/a-decrease-in-the-coverage-percentage-is-not-a-reliable-metric/2621 Do we want to handle this naively (i.e. fail if global absolute coverage decreases) and see whether we run into the problems discussed in the thread RE: deleted code, or should we start with a more sophisticated design off the bat? I like the idea of measuring the coverage of new code and/or measuring a change in the number of lines not covered, but I don't see an easy way to implement either of those approaches with covr::package_coverage.

dfsnow commented 4 weeks ago

@jeancochrane Let's go ahead with counting lines. covr::package_coverage spits out an object that (I think) you can feed to covr::tally_coverage to get line counts. This feels like a good jr project.

jeancochrane commented 4 weeks ago

Sounds good @dfsnow! Unassigning myself and putting it back On Deck. Here's some more context for whoever picks this up next:

By "a simple coverage badge with a check that fails if coverage is reduced", we mean that we should remove the codecov badge from READMEs with a test-coverage workflow, and instead use only the test-coverage badge (see the README for lightsnip for an example of a project that currently displays both badges). Since the test-coverage badge displays the status of the most recent run of the test-coverage GitHub workflow on the main branch, we'll need to refactor that logic when we factor it out into a composite action so that it executes the following steps:

  1. Pulls the number of lines not covered in the previous run on the main branch from an AWS S3 bucket
  2. Counts the number of lines that are not covered in the current version of the code (main branch or PR)
  3. If the number of lines covered has increased: Fails the workflow
  4. If the number of lines covered has not increased, and the workflow is running on the main branch: Pushes the new number of lines not covered to the remote storage location in the AWS S3 bucket

If someone picks this up who is not me, let me know when you get started and I can help get the AWS S3 infra up and running.