Kotlin / kotlinx-kover

Apache License 2.0
1.25k stars 46 forks source link

Execute tests and kover separately (Gradle-plugin) #565

Open Ragin-LundF opened 2 months ago

Ragin-LundF commented 2 months ago

In our build pipeline, jUnit tests and Cucumber tests are separated. The coverage report should basically be created afterwards.

Is there a way to configure the Gradle plugin so that it only creates the report but does not run the tests again?

I could do this via CLI with gradle koverXmlReport -x test -x cucumber.

However, we also use a build library that controls the tasks. If I now have to add all the excludes per project, this is always an unnecessary configuration effort that can quickly go wrong.

Thanks!

shanshin commented 2 months ago

Hi, is build cache applicable for your project?

Ragin-LundF commented 1 month ago

Build cache is not a solution in this case, as the tests still run twice, which may extend the pipeline for several hours.

The pipeline always has one step for the jUnit tests, a second for the Cucumber tests and there may also be various E2E tests in Cucumber.

I would like to be able to control when which tests are called.

It is also a massive problem that the tasks are automatically attached to the build task. This means that a pure build must also be executed with -x test -x cucumber -x .....

A simple configuration of which tasks should be attached to would be much better.

If these tasks are then executed, the agent should also be attached, but should not affect the other dependencies.

shanshin commented 1 month ago

Build cache is not a solution in this case, as the tests still run twice, which may extend the pipeline for several hours.

If using build cache and no sources changed tests shouldn't execute again.

shanshin commented 1 month ago

The pipeline always has one step for the jUnit tests, a second for the Cucumber tests and there may also be various E2E tests in Cucumber.

In the Kover Gradle Plugin, you can build a report only on completed tests or taken from the cache. Building the report later in a separate build requires manual configuration and use, for example, Kover CLI.

For a custom pipeline, you may use the Kover instrumentation agent and independently instrument the necessary tasks, collect binary reports from different builds and generate human-readable reports on them by Kover CLI.

shanshin commented 1 month ago

Also, in Kover Gradle Plugin you can disable running of some test tasks by their name. In 0.7.x

kover {
    excludeTests { 
        tasks("cucumber") // or "test"
        // you may write something like 
       // if (hasProperty("disableCuCumber")) tasks("cucumber")
    }
}

In 0.8.0-Beta

kover {
    currentProject {
        instrumentation {
            disabledForTestTasks.addAll("cucumber")
        }
    }
}

But in this case, you cannot generate a readable report (XML or HTML), because you do not run the second task. Therefore, you can call the generation of a binary report (koverBinaryReport task) and use it in the CLI.