Support native execution of JUnit tests via Graal's NativeImageJUnitLauncher

[dthommes]

Description

The Graal Team has announced support for running JUnit 5 tests natively in June:

https://medium.com/graalvm/gradle-and-maven-plugins-for-native-image-with-initial-junit-testing-support-dde00a8caf0b

In short, it works like this:

• A native-image feature (JUnitPlatformFeature) collects information about the JUnit tests in the classpath at build time.
• A native image executable is built containing all production and test code including a special JUnit Runner (NativeImageJUnitLauncher).
• When starting the executable, the NativeImageJUnitLauncher (main) executes the tests.

This technique is amazing, as many projects need a good code coverage to make sure, natively compiled code behaves the same as Java code. Large projects with existing unit tests can also be migrated with more ease, as there is not such a hard requirement for integration test coverage.

Findings so far:

• The recommended Maven and Gradle Plugins by the Graal team cannot be combined with the Quarkus plugin easily (NPEs coming up).
• Probably, the Quarkus plugins should support the native build and execution of the unit tests.
• It is not possible to use the @QuarkusMain approach, as the test classpath is missing, even if we point to a different main method.
• It is not possible to manipulate the command line for the native-image compiler via the Quarkus plugins to have it use the NativeImageJUnitLauncher as main class. This is due to Quarkus' current approach of feeding a prepared JAR (including manifest) to the compiler with the -jar parameter.

Implementation ideas

In my opinion, the most useful approach would be to have new phases/tasks in the Quarkus plugins allowing the build and execution of JUnit tests. A corresponding manipulation of the required native-image compile command is documented here.

https://github.com/graalvm/native-build-tools/blob/master/common/junit-platform-native/README.md

The following is an example command line from a test project, where I was able to manually build a native executable using NativeImageJUnitLauncher as main class. Please note, that for this approach to work, I had to move all tests into the main folder and set all test dependencies to the compile scope to make sure, the test classpath is available.

/Users/thommes/.sdkman/candidates/java/21.2.0.r11-grl/bin/native-image \
-J-Djava.util.logging.manager=org.jboss.logmanager.LogManager \
-J-Dsun.nio.ch.maxUpdateArraySize=100 \
-J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory \
-J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 \
-J-Duser.language=de -J-Duser.country=DE -J-Dfile.encoding=UTF-8 \
-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime \
-H:+JNI -H:+AllowFoldMethods -cp native-junit-1.0.0-SNAPSHOT-runner.jar \
-H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http \
-H:-UseServiceLoaderFeature -H:+StackTrace -H:-ParseOnce \
-H:Path=/projects/native-junit/target/native-junit-1.0.0-SNAPSHOT-native-image-source-jar \
-H:Name=native-tests \
--features=org.graalvm.junit.platform.JUnitPlatformFeature \
--no-fallback \
org.graalvm.junit.platform.NativeImageJUnitLauncher

Please note the use of -cp instead of -jar and the alternative main class being mentioned at the end.

This manual approach did not include the agent-based execution of the tests, that is currently supported by the Graal plugins to collect configuration information. This might be an additional feature, that could iteratively be added.

[mkouba]

A native image executable is built containing all production and test code including a special JUnit Runner...

Hm, but in that case you're not testing the production native image anymore. So you'd need to build two native images when running the build, i.e. production and test, or?

[dthommes]

A native image executable is built containing all production and test code including a special JUnit Runner...

Hm, but in that case you're not testing the production native image anymore. So you'd need to build two native images when running the build, i.e. production and test, or?

Yes. You would have a twofold test strategy:

• Test your production build with integration tests
• Additionally run unit tests with the test executable

With this approach, you get better test coverage, because the unit tests allow better coverage of the edge cases. It is more effort, but can typically done within a build pipeline.

[quarkus-bot[bot]]

/cc @geoand