Respect `Run Tests Using` setting

[sandra-thieme]

I have Intellij configured to run tests with gradle (Settings > Build, Execution, Deployment > Build Tools > Gradle > Run Tests Using).

It seems like this setting is not yet picked up by the kotest plugin. I've checked the Kotest run configurations, but it doesn't look like I can do anything with gradle there. Some of our tests can now not be run in the IDE because they rely on configuration in gradle.

[sksamuel]

I've added a reply to the other ticket.

[pschyska]

@sksamuel The static properties support doesn't help us because our gradle build sets up dynamic properties around test tasks (e.g. kubernetes endpoints for integration tests). IntelliJ is set to run tests via gradle and running a test class does a gradle test --tests=..., which is fine. But running a single test or container fails: A gradle build is triggered, but the plugin then runs a separate java command which of course loses the environment variables we set up. Is there a way to dynamically inject the properties, or make the plugin use gradle for running in that case as well?

[sksamuel]

I guess in theory we could make the forked process propagate properties too. Using gradle for a single test won't work properly until a later version as gradle doesn't support nested tests properly.

[pschyska]

I guess in theory we could make the forked process propagate properties too.

That would be really helpful. Right now we are using environment variables -- they are easier to manage when the apps are containerized and running in Kubernetes, for example. But we could switch to system properties or make our code use either. I had another idea: Where does kotest look for properties? I could make gradle's test task render one on each run, then the forked java process would hopefully be able to pick them up.

Using gradle for a single test won't work properly until a later version as gradle doesn't support nested tests properly.

Do we need special gradle support here? The plugin could run something like gradle test --tests=MyTestClass -Dsome.kotest.specific="some test name", I assume?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[pschyska]

@sksamuel From your suggestion I built a solution that generates a a kotest properties file and adds it to the test classpath. It's pretty unobvious though, and the default behaviour in the IDE is confusing: Running a test class is run as a normal gradle task with all the riders attached, and running a single test runs AFAICS the testClasses goal in gradle, and then the tests as a separate process. Additionally, we have multiple test configurations beside test, so testClasses is not always the right thing to run. Accordingly, running a test class in the IDE asks which configuration to run, which doesn't happen when running some scope within a class. We have basically 3 configurations: test (unit + integration tests), unitTest (just unit tests) and systemtest (black box tests against a locally deployed full system). How hard would it be to re-use the IDE logic that applies when running a class to a contrained scope? In my mind, the plugin could do whatever the IDE is doing, and attaching a textual filter on the scope (kotest.filter.tests or kotest_filter_tests) ?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[OliverO2]

The inability to set jvmArgs makes it currently impossible to run BlockHound-enabled tests on JDK versions >= 13 via the plugin. BlockHound#33 requires the following workaround:

tasks.withType<Test>().configureEach {
    // WORKAROUND: https://github.com/reactor/BlockHound/issues/33 required for JDK >=13
    if (JavaVersion.current().isCompatibleWith(JavaVersion.VERSION_13)) {
        jvmArgs = listOf("-XX:+AllowRedefinitionToAddDeleteMethods")
    }
}

[ulrikrasmussen]

We would also be very interested in a fix for this. We currently have to resort to prepending f: to tests and running the whole suite, but this is much less convenient than being able to just run the test case directly. Developers who are new to the code base also frequently trips over this behavior when trying to run individual tests and the expected environment configured by Gradle is missing.

[OliverO2]

@ulrikrasmussen Just a hint that does not solve the original problem, but may help in your use case:

You could use test filtering in the build script instead of prepending "f:" to individual tests:

tasks.withType<Test>().configureEach {
    filter {
        listOf<String>(
            // "serialization.CommonSerializationTests*",
            "serialization.JvmSerializationTests*",
        ).forEach {
            includeTestsMatching("$it")
        }
    }
}

Might be easier with groups of tests and/or running tests across multiple subprojects.