Is JUnit only supported testing provider -- even excluding other JUnit 5 `TestEngine` compatible providers?

[GavinRay97]

Description

I would like to help add support for Kotest to Quarkus:

• https://github.com/kotest/kotest
• https://github.com/kotest/kotest/issues/1401

Here's an example project:

• https://github.com/GavinRay97/kotest-examples-quarkus/tree/feature/update-deps

---

In trying to debug why this isn't working, it looks like:

• CDI injection doesn't work outside of junit.jupiter tests, maybe due to annotation processing?
• The DevMode test provider explicitly only looks for junit annotations:

https://github.com/quarkusio/quarkus/blob/0580569c85cc814a0bc9b4107686928d8d1c9896/core/deployment/src/main/java/io/quarkus/deployment/dev/testing/JunitTestRunner.java#L644-L654

Where IE, PARAMATERIZED_TEST here is:

import org.junit.jupiter.params.ParameterizedTest;
public static final DotName PARAMETERIZED_TEST = DotName.createSimple(ParameterizedTest.class.getName());

---

When running quarkus:dev/quarkusDev it doesn't detect the Kotest tests. When running manually, there are injection/provider failures:

import io.kotest.core.spec.style.AnnotationSpec
import io.kotest.matchers.shouldBe
import io.quarkus.test.junit.QuarkusTest
import javax.inject.Inject

@QuarkusTest
class QuarkusUserServiceTest : AnnotationSpec() {

   @Inject
   lateinit var userService: UserService

   @Test
   fun `user service should be injected`()  {
      userService.repository.findUser().name shouldBe "system_user"
   }
}

lateinit property userService has not been initialized
kotlin.UninitializedPropertyAccessException: lateinit property userService has not been initialized
    at io.kotest.examples.quarkus.QuarkusUserServiceTest.getUserService(QuarkusUserServiceTest.kt:12)
    at io.kotest.examples.quarkus.QuarkusUserServiceTest.user service should be injected(QuarkusUserServiceTest.kt:16)

Looking at the deps, they seem to be in sync with the same ones quarkus-junit is already using, so I'm not certain why it doesn't work, or if the infra is even set up atm to handle non-junit (even if JUnit compatible) providers?

Implementation ideas

No response

[quarkus-bot[bot]]

/cc @evanchooly

[famod]

/cc @stuartwdouglas

[stuartwdouglas]

How is this supposed to work? When I run the gradle build I just see gradle running the standard JUnit test.

Continuous testing won't pick this up because it is currently using the JUnit test annotations for discovery, however that could be changed if needed.

[GavinRay97]

If you mean, in the sense of "how would it work in general" -- I believe that Kotest aims for transparent JUnit5 support.

The way to add and use Kotest in a project, is by enabling JUnit as the testing provider for instance, and just having the Kotest runner dep in the classpath. Then it should "just work":

• https://kotest.io/docs/quickstart/#test-framework

Sam (Kotest maintainer) has commented before that he believes after looking at Quarkus' JUnit implementation, there shouldn't be anything preventing it from working, except possibly supporting InvocationInterceptor:

• https://github.com/kotest/kotest/issues/1401#issuecomment-719008136
• https://github.com/kotest/kotest/blob/master/kotest-extensions/kotest-extensions-junit5/src/jvmMain/kotlin/io/kotest/extensions/junit5/KotestExtension.kt

I'm no expert, I've asked @sksamuel if he'd be able to join the conversation here, but I think he's currently on vacation 🌴 (back later in week). He would most certainly have better answers, this is the best I can come up with based on what I think you're asking.

Let me know if this isn't what you meant 😅

[sksamuel]

So my understanding (and it's been a while since I looked) - is that the testing support for JUnit in Quarkus is built using a bunch of JUnit extensions, and you do the compilation magic as part of the junit lifecycle.

My guess is that as long as Kotest can map its own lifecycle callbacks into the junit ones, then we should be able to just use the junit rule. The caveat being that junit is built around methods in classes, and kotest is a modern functional based testing framework where anything goes so it might not map 1 to 1.

Really what would be ideal is if Quarkus separated its testing magic from the JUnit extensions and added its own interface in the middle. So then any test framework could hook in its own callbacks into the right place.

[sksamuel]

Kotest could have its own version of this: https://github.com/quarkusio/quarkus/blob/4b728c83684547e3c5b8fa54933f23c9d13e53a8/test-framework/junit5/src/main/java/io/quarkus/test/junit/QuarkusTestExtension.java

But at 1300 lines of code, that's unlikely to happen by an someone who isn't a quarkus expert! So I think the quickest path forward would be to hook each of those junit extensions into a kotest extension (I can add any missing ones) and then cross our fingers it works :)

[GavinRay97]

My guess is that as long as Kotest can map its own lifecycle callbacks into the junit ones, then we should be able to just use the junit rule. The caveat being that junit is built around methods in classes, and kotest is a modern functional based testing framework where anything goes so it might not map 1 to 1.

Maybe we could start by limiting the implementation of the Quarkus <-> Kotest functionality to JUST the AnnotationSpec, since that will behave identically to plain JUnit @Tests?

That seems like it might be the "easiest" to get running properly since it at least maps 1-to-1 with JUnit behavior?

This weekend I will return to the Kotest + Quarkus repo and minimize the implementation, got caught up at office this week.

[antoniomacri]

I'm also very interested in Quarkus + Kotest.

Any progress on this? Is there anything I can do to help? I'm no expert in Quarkus extensions or Kotest, but I'd like to help.

JUnit architecture seems fairly complex to me, so I don't know which is the correct way to integrate Quarkus and Kotest.

Two ways I can think of.

1. As far as I understand the suggested approach by @sksamuel is: rewrite QuarkusTestExtension by attaching not to JUnit lifecycle callbacks, but to Kotest ones. @sksamuel @GavinRay97 anyone already tried/found problems?

2. I'm also thinking about a sort of @TestFactory as @stuartwdouglas did for Cucumber (https://github.com/quarkiverse/quarkus-cucumber/blob/main/runtime/src/main/java/io/quarkiverse/cucumber/CucumberQuarkusTest.java). Would it be possibile for Kotest to process the specs and produce a representation like DynamicTests, which can then be integrated in a quarkus-kotest extension (like for Cucumber)?

PS: can the title of this issue be changed to something more explicit like "Integrating Kotest"?