Kotlin / kotlinx-kover

Apache License 2.0
1.37k stars 53 forks source link

Expected coverage with variable threshold #545

Closed realdadfish closed 6 months ago

realdadfish commented 9 months ago

What is your use-case and why do you need this feature?

Configuring the boundaries of coverage checks, i.e. minBound and maxBound, is quite cumbersome, as they're "moving targets" as the project continues to be developed. Also, the corridor in which boundary checks are useful to be applied changes over time. A module with 1000 lines of code should of course have different boundaries than a module with 10.000 lines of code.

Describe the solution you'd like

Ideally, instead / in addition to the regular way of defining boundaries I'd like to be able to configure boundaries based on the number of lines of code in a module (Java and Kotlin), something like this:

verify {
    rule {
        bound(KoverThresholdBound(50) { loc: Int ->
             when {
                 loc < 1000 -> 10
                 loc < 5000 -> 5
                 loc < 10000 -> 2
                 default -> 1
        })
    }
}

So, following this example, the actual minBound and maxBound would be calculated like this:

val minBound = (expected - threshold).coerceAtLeast(0)
val maxBound = (expected + threshold).coerceAtMost(100)

The callback could also be shared by some custom build-convention plugin, so that one would only need to configure the expected coverage (50 in this case) per module.

shanshin commented 9 months ago

This solution is suitable only for a specific case. For custom verification logic, logging, etc., I think it's better to add the execution of an action that would take the original coverage values:

E.g.

kover {
    ...

    coverage(MetricType.LINE, GroupingEntityType.APPLICATION) { coverage -> 
        // this action will be executed only when the coverage is evaluated
        if (coverage.totalCount > 0) {
            val coverageRate = coverage.coveredCount / coverage.totalCount
            if (coverageRate < 0.5) {
              throw Exception(" coverage for ${coverage.entityName} is below 50%: $coverageRate")
              // or log
            }
        }  
    }
   ...

}
shanshin commented 6 months ago

Closed in favor of #605