[koverVerify] Projects with no tests are skipped

[alexandrefavre4]

Hi there,

I configured a 80% line coverage rule on some subprojects. Among these subprojects, some don't have any tests but koverVerify terminates successfully because Kover is skipping them: Task 'koverVerify' will be skipped because no tests were executed

As the 80% line coverage rule has not been met for these subprojects, I would have expected koverVerify to fail.

Did I miss an option that would make it fail in this situation? If not, is this a change you would consider making to Kover?

Cheers!

[shanshin]

Hi, if no dependency is specified, then only those tests that are declared in the same project are used for calculating coverage.

When there is cross-testing between subprojects, it is recommended to use a combined report on several projects.

[alexandrefavre4]

@shanshin I'm not cross-testing between subprojects. I've configured the rule to be applied to each subproject independently. I get the same result if I run koverVerify on a single subproject (./gradlew :mysubproject:koverVerify). Kover is still skipping verification as mysubproject has no tests.

[shanshin]

Kover is still skipping verification as mysubproject has no tests.

Kover uses only tests that are declared in the same project, and does not know in advance whether any other projects can test his classes.

To do this, add a dependency

dependency {
    kover(project(":projectTestingMyClasses"))
}

so that when executing reports, the results of running tests from specified projects are taken into account.

As a side effect, the report will take into account classes from the specified dependency. If there is such a possibility, then these classes can be filtered by package.

[alexandrefavre4]

@shanshin It has nothing to do with subprojects.

Let's take the example app minimal and add the following configuration to the build.gradle.kts:

koverReport {
    defaults {
        verify {
            rule {
                bound {
                    minValue = 80
                    metric = kotlinx.kover.gradle.plugin.dsl.MetricType.LINE
                    aggregation = kotlinx.kover.gradle.plugin.dsl.AggregationType.COVERED_PERCENTAGE
                }
            }
        }
    }
}

Then, running gradle koverVerify should print you the following output:

> Task :koverVerify FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':koverVerify'.
> A failure occurred while executing kotlinx.kover.gradle.plugin.tools.kover.VerifyReportAction
   > Rule violated: lines covered percentage is 57.142900, but expected minimum is 80

Now, if you delete the test folder and rerun gradle koverVerify, the task will complete successfully:

> Task :koverVerify SKIPPED
Task 'koverVerify' will be skipped because no tests were executed

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/8.1.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 673ms

We can see in the output that the task was skipped because there were no tests to be executed. In my opinion, koverVerify should fail in this situation, as the coverage didn't reach 80%.

[shanshin]

Perhaps a design is needed on the question of what to do if there are no tests in the project.

[alexandrefavre4]

Yes, I think that would make sense.

[quanturium]

+1 to being able to configure what to do when tests are not there. In our use case, we want a report to show 0% coverage when nothing is there so that we can post-process and track those over time.

[shanshin]

Closing the issue as a duplicate, you can follow the progress of the https://github.com/Kotlin/kotlinx-kover/issues/409 task.