Consider offering artifacts that support the Arquillian Junit5 runner

[KyleAure]

Issue Overview

The current Open Liberty Arquillian artifacts for JUnit:

io.openliberty.arquillian:arquillian-liberty-managed-junit:1.1.8
io.openliberty.arquillian:arquillian-liberty-remote-junit:1.1.8
io.openliberty.arquillian:arquillian-liberty-remote-jakarta-junit:2.1.0
io.openliberty.arquillian:arquillian-liberty-managed-jakarta-junit:2.1.0

All have a common dependency on

org.jboss.arquillian.junit:arquillian-junit-container:1.6.0.Final

More modern tests are being written using Junit5 which require the Arquillian Junit5 Runner:

org.jboss.arquillian.junit5:arquillian-junit5-container:1.7.0.Alpha13

Expected Behaviour

Liberty Arquillian should support the JUnit 5 Runner

Current Behaviour

As-is it does not seem as if Liberty Arquillian supports the JUnit5 Runner

Additional Information

Current workaround:

<dependencyManagement>
    <dependencies>
    <dependency>
        <groupId>org.jboss.arquillian</groupId>
        <artifactId>arquillian-bom</artifactId>
        <version>1.7.0.Alpha13</version>
        <type>pom</type>
        <scope>import</scope>
    </dependency>
    <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>5.9.0</version>
        <type>pom</type>
        <scope>import</scope>
    </dependency>
    </dependencies>
</dependencyManagement>
<dependencies>
    <dependency>
    <groupId>io.openliberty.arquillian</groupId>
    <artifactId>arquillian-liberty-managed-jakarta-junit</artifactId>
    <version>2.1.0</version>
    <type>pom</type>
    <!-- Use Junit5 Runner instead-->
    <exclusions>
        <exclusion>
            <groupId>org.jboss.arquillian.junit</groupId>
            <artifactId>arquillian-junit-container</artifactId>
        </exclusion>
    </exclusions>
    </dependency>
    <dependency>
    <groupId>org.jboss.arquillian.junit5</groupId>
    <artifactId>arquillian-junit5-container</artifactId>
    </dependency>
</dependencies>

[cherylking]

@KyleAure I took a look at this last week and it seems that it is not a straight forward change. The test cases in https://github.com/OpenLiberty/arquillian-liberty-dependencies fail after making the change, and my first couple of attempts to resolve the failures did not work. The @RunsWith is no longer available and neither is the org.jboss.arquillian.junit.Arquillian used in the @RunsWith.

If you have some insight as to the changes needed in the testcases in that repo (which is also where the Junit runner would need updating), please let me know.

One other point, I don't think we would want to make this change for the non-Jakarta artifacts.

[KyleAure]

@cherylking For testing, we will need to have a separate test package and it will need to run in a separate JVM. Junit5 has a vintage Junit4 runner which can be used to run the Junit4 test classes. But the problem is the Arquillian container, only one container can be loaded per JVM.

Long story short: Junit5 supports running Junit4 tests, but the Arquillian Junit5 Container cannot run Junit4 Container tests.

The @RunsWith is no longer available and neither is the org.jboss.arquillian.junit.Arquillian used in the @RunsWith.

The proper way to annotate Junit5 tests is with extensions (instead of runners):

@ExtendWith(ArquillianExtension.class)
public class WLPJunit5TestCase {
}

One other point, I don't think we would want to make this change for the non-Jakarta artifacts.

I agree

[KyleAure]

Options:

1. Refactor all existing tests to use JUnit5
2. Keep liberty-managed tests using JUnit4, and Refactor liberty-remote to use JUnit5
3. Create a subproject inside liberty-managed and liberty-remote that tests JUnit5 (Need to duplicate tests or write simple tests)
4. Don't automate testing for JUnit5, and release a pom artifact for Junit5 for manual testing (risky).