Test framework agnostic test utils

[sksamuel]

Description

Quarkus currently has an excellent unit test support via the JUnit test extensions, but these are not possible (or at least difficult) to adapt to other test frameworks, such as the Kotlin based kotest.

What would be great is if the test extension was abstracted so instead of relying on JUnit specific interfaces such as InvocationInterceptor, it relied only on lifecycle interfaces defined by Quarkus.

Implementation ideas

No response

[quarkus-bot[bot]]

/cc @evanchooly

[snowe2010]

Would love to have support for this. We've been wanting to use kotest on several teams at Sunrun for a few years now and the kotest team has been unable to implement it due to the tight connection between JUnit and Quarkus. Any help here would be appreciated.

[famod]

Looks very related to #19749? /cc @geoand @stuartwdouglas

[sksamuel]

I would say it is related, but I wouldn't aim for just JUnit compatible engines, but any test framework. If you can abstract the lifecycle events you need to things like "configure a test class", "before a test", "after a test" and so on, then any test engine should be able to integrate.

[stuartwdouglas]

It looks like Kotest is also JUnit based, so in theory it should be possible to integrate it. I don't think it is possible to provide all this in a generic way though, as it really depends on how the test framework is implemented (e.g. when we supported JUnit 4 there was a huge difference in how the two integrations worked, just based on the API's provided by the framework).

If you did want to make this work I would look at doing up a kotest module by using the existing JUnit extension as a base, and from there see if there is common code that can be shared. I think quarkus-kotest would be a good candidate for a Quarkiverse extension.

[sksamuel]

I think the difference is that Quarkus uses JUnit Jupiter, not JUnit platform. So jupiter is a specific test engine implementation, and that's why we can't just plugin another junit platform compatible engine.

I don't know Quarkus well enough to opine on the easiness of adding a generic test layer, but my assumption (possibly wrong), is that you need things like "do this before a test", "do this to setup a class' which can be done abstractly, rather than concretely tied into the junit lifecycle interfaces.

A bit like how spring TestContextManager has beforeTestClass and afterTestClass and you just need to plumb them into whatever your test framework calls the equivalent things.

[stuartwdouglas]

The quarkus-junit module it build on junit-jupiter, but there is no reason why you could not add a quarkus-kotest that is not tied to Jupiter.

Continuous testing does have a dependency on Jupiter, but it can still work with other engines, and the dependency on Jupiter can likely be removed if required, as it is only really used to support tags.

[sksamuel]

I guess we can try :) I did try in the past, but I thought the junit module was quite complicated for someone who doesn't work on the quarkus codebase.

[geoand]

I don't think it is possible to provide all this in a generic way though, as it really depends on how the test framework is implemented

I very much agree