IntelliJ NativeLibraryProvider is missing for Java 21

[ChrisHegarty]

The following exception is often encountered with running tests with the IntelliJ test runner. Running tests with the Gradle runner is fine.

java.lang.LinkageError: NativeLibraryProvider is missing for Java 21

    at org.elasticsearch.nativeaccess.lib.NativeLibraryProvider.loadJdkImpl(NativeLibraryProvider.java:85)
    at org.elasticsearch.nativeaccess.lib.NativeLibraryProvider.loadProvider(NativeLibraryProvider.java:67)
    at org.elasticsearch.nativeaccess.lib.NativeLibraryProvider$Holder.<clinit>(NativeLibraryProvider.java:97)
    at org.elasticsearch.nativeaccess.lib.NativeLibraryProvider.instance(NativeLibraryProvider.java:43)
    at org.elasticsearch.nativeaccess.NativeAccessHolder.<clinit>(NativeAccessHolder.java:22)
    at org.elasticsearch.nativeaccess.NativeAccess.instance(NativeAccess.java:24)
    at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:282)
    at org.elasticsearch.bootstrap.BootstrapForTesting.<clinit>(BootstrapForTesting.java:91)
    at org.elasticsearch.test.ESTestCase.<clinit>(ESTestCase.java:329)
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:534)
    at java.base/java.lang.Class.forName(Class.java:513)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:631)
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.nativeaccess.jdk.JdkNativeLibraryProvider

[elasticsearchmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[mark-vieira]

@rjernst I assume this is related to all the refactoring around native/jna stuff?

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[rjernst]

This is a limitation of IntelliJ. Since the jdk21 provider is loaded through an MRJAR, it won't work in intellij. Test must be run either with JDK 17, or with the gradle runner.

[mark-vieira]

So is this just that the additional sourcesets aren't on the runtime classpath when running in intellij? Could we hack this somehow?

[mark-vieira]

Or more simply, given that the new minimum runtime version is Java 21, couldn't we collapse the MR sources down into the main sourceset? For 9.0 the pre-21 implementations will never be used so why separate that in the MR jar> We should only use the MR jar for Java 22+ impls.

[rjernst]

So is this just that the additional sourcesets aren't on the runtime classpath when running in intellij? Could we hack this somehow?

Unfortunately adding the additional sourcesets in a non MRJAR means there is jarhell. So we would have to relax our jarhell check, and then also add special handling to know how to fallback to the jdk 17 impl when not running on jdk 21+. It would be complicated, and I don't think worth it (we discussed when first adding native access).

given that the new minimum runtime version is Java 21, couldn't we collapse the MR sources down into the main sourceset

Yes, I plan to do this now that main is Java 21.

[ChrisHegarty]

Is there any way to configure Gradle (or the import of our Grdle project into IntelliJ) so that tests depend upon the jar file of the native subproject, rather than the exploded classes? This would be much closer to actual runtime behaviour.

[breskeby]

@ChrisHegarty for running those tests with gradle and the gradle runner in idea this should be possible

[mark-vieira]

@ChrisHegarty for running those tests with gradle and the gradle runner in idea this should be possible

Right, we can configure Gradle to do this if we want (and in fact we do for MR jars I believe) but the root issue here is with the IDE so using the Gradle running kind of defeats the purpose.

Collapsing the source sets as Ryan says will solve this issue so I don't think we need to do anything else.