Kotlin / kotlinx-kover

Apache License 2.0
1.28k stars 48 forks source link

excludes package and patterns is not working at all #80

Closed periva101 closed 2 years ago

periva101 commented 2 years ago

non of the following is working

testOptions {
        unitTests.all {
            kover {
                enabled = true
                includes = ['com\\.benoholding\\..*']
                excludes = [
                        'com\\.genoholding\\.geno\\.data\\.model\\..*',

                ]
            }
        }
    }
shanshin commented 2 years ago

Hi, could you please clarify what type of project you have (Kotlin/JVM, Kotlin Multiplatform, Android) and what Coverage Engine is used (IntelliJ, JaCoCo)? A small reproducer or a link to the project would also help a lot.

shanshin commented 2 years ago

Relates #65

periva101 commented 2 years ago

I found that following to kover doc 'com\.genoholding\.geno\.data\.model\..', does not work but com.genoholding.geno.common. is working fine

SimonedeGijt commented 2 years ago

Hey guys, I have troubles with excluding my testclasses from the report. First question would be; why is this even included in the first place?! Seems to be related to the fact that other test-sources like itest or contractTest are not recognised for being test code. Related to #83

shanshin commented 2 years ago

I found that following to kover doc 'com.genoholding.geno.data.model..', does not work but com.genoholding.geno.common. is working fine

Now if you use INTELLIJ as an Coverage Engine, you should write regular expressions as a filter. e.g. com\\.genoholding\\.geno\\.data\\.model\\..*.

However, in the next release, for the filters will be used the same format as for JaCoCo, with wildcards ? and *.

SimonedeGijt commented 2 years ago

I found that following to kover doc 'com.genoholding.geno.data.model..', does not work but com.genoholding.geno.common. is working fine

Now if you use INTELLIJ as an Coverage Engine, you should write regular expressions as a filter. e.g. com\\.genoholding\\.geno\\.data\\.model\\..*.

However, in the next release, for the filters will be used the same format as for JaCoCo, with wildcards ? and *.

In our case the issue is not really about the regex but about the source sets it is looking at. So it's not looking inside all our source sets.

shanshin commented 2 years ago

I found that following to kover doc 'com.genoholding.geno.data.model..', does not work but com.genoholding.geno.common. is working fine

Now if you use INTELLIJ as an Coverage Engine, you should write regular expressions as a filter. e.g. com\\.genoholding\\.geno\\.data\\.model\\..*. However, in the next release, for the filters will be used the same format as for JaCoCo, with wildcards ? and *.

In our case the issue is not really about the regex but about the source sets it is looking at. So it's not looking inside all our source sets.

These filters are not filters for the report, they determine which classes should be instrumented by the agent (estimate the coverage for them). Filters for reports are currently in development (#17).

In this case, to exclude a class from the report, you need to exclude it from all test tasks in your project. Excluding a class in only one test task may not have an effect, since this class can be instrumented by another task and so will get into the report.

SimonedeGijt commented 2 years ago

Hey @shanshin, I'm not sure that I understand what you mean. I don't really know what that would look like in practise. Currently I have my configuration set like this:

withType<Test> {
        useJUnitPlatform {
            excludeTags("smoke")
        }

        extensions.configure(KoverTaskExtension::class) {
            includes = listOf("org\\.example\\..+")
            excludes = listOf(
                ".+Test",
            )
        }
    }

I made an example project to show what happens. As you can see the GreetingTest (which is part of itest) is still included in the report, whereas GreetingServiceTest (part of test) is not.

example-kover-tests.zip

shanshin commented 2 years ago

Hey @shanshin, I'm not sure that I understand what you mean. I don't really know what that would look like in practise. Currently I have my configuration set like this:

withType<Test> {
        useJUnitPlatform {
            excludeTags("smoke")
        }

        extensions.configure(KoverTaskExtension::class) {
            includes = listOf("org\\.example\\..+")
            excludes = listOf(
                ".+Test",
            )
        }
    }

I made an example project to show what happens. As you can see the GreetingTest (which is part of itest) is still included in the report, whereas GreetingServiceTest (part of test) is not.

example-kover-tests.zip

Thanks for the example! In fact, coverage for the class GreetingTest itself is not present in the report, however, due to the fact that lambda functions are present in it, nested classes are created for them, which fall into the report. In order to exclude them, you can write the following

        extensions.configure(KoverTaskExtension::class) {
            includes = listOf("org\\.example\\..+")
            excludes = listOf(".*Test\\$.+", ".+Test")
        }

Unfortunately, at the moment the Kover plugin does not provide to specify which source set to exclude from the instrumentation, it excludes only the source set with the name test (this feature will be added later). Therefore, I would recommend putting the test classes in a special package that can be completely excluded, for example, org.example.itest and configure

        extensions.configure(KoverTaskExtension::class) {
            excludes = listOf("org\\.example\\.itest\\..+")
        }
shanshin commented 2 years ago

@periva101, have you tested version 0.5.0-RC2?

SimonedeGijt commented 2 years ago

Hey @shanshin, do you have a reference to this feature you are talking about: "Unfortunately, at the moment the Kover plugin does not provide to specify which source set to exclude from the instrumentation, it excludes only the source set with the name test (this feature will be added later)."

I would like to subscribe myself to it if possible :)

shanshin commented 2 years ago

Hey @shanshin, do you have a reference to this feature you are talking about:

Now there is no separate task for this, because the API design has not been finished yet. You can look at #19, there will be added links to individual tasks for implementation: there are plans to add tasks for source sets and verification rules.