Feature Request: Get coverage since specified commit target

[tomdaniel-it]

I see this functionality was already asked about in #1148 , but since this issue was closed, I thought I'd open a Feature Request for it.
This issue was also created as reaction to performance issues. While this is also an advantage, my feature request focuses on other benefits that this feature would introduce.

Feature request: An extra option to define a commit target to compare against, so that only for the changes since this commit, coverage will be created. This would mainly be used in CI/CD pipelines.

Why: In greenfield projects, it is easy to setup the constraint of having a minimum % of coverage, otherwise the build fails and you can't merge your pull requests. But in reality, this isn't as easy. Most of the times developers are working with already existing code bases which are not covered by unit tests. The only options are:

• Add a ton of unit tests to reach your minimum % coverage
  • Most organizations will not accept the cost of this
• Exclude all untested code from coverage
• Disable the minimum % coverage constraint in your pipelines, and only use the coverage for looking at it manually

This feature would introduce two new benefits (besides performance):

• Being able to add minimum % coverage constraints to untested projects, so that changes & additions are required to be tested
  • This would motivate a lot of code projects to introduce coverage as part of their pipeline, as a migration phase to hopefully eventually a fully tested codebase
• The % coverage will barely be impacted by untested changes on a large codebase, making it possible for untested code to get through the minimum % coverage constraints. With this feature, this issue could be resolved.

This feature request was inspired by the since option flag in .NET Stryker.

[Bertk]

Hi, thank you for the request but comparing code coverage of SCM branches is not in scope of coverlet but is available in CI extensions or code quality assurance tools.

e.g.

• SonarQube
  • Branch analysis for long-living (releases) and short-lived branches (PRs)
  • tests -Coverage on new code: This definition is identical to Coverage but is restricted to new/updated source code.
• GitHub
  • Using GitHub Actions to Help Review Pull Requests Using Unit Tests and Code Coverage
  • SonarQube Quality Gate check
  • Azure DevOps
  • Quality Gate: Configuring Code Coverage Checks for .Net Core Microservices in Azure DevOps
  • Enforce Code Coverage as Part of Pull Requests in Azure DevOps

[github-actions[bot]]

This issue is stale because it has been open for 3 months with no activity.