Open jerboaa opened 10 months ago
Going by the discussion in https://github.com/oracle/graal/discussions/8012, specifically this:
Foreign Function and Memory API support (OpenJKD Project Panama): Since support in native image
is still under development, it needs to be enabled explicitly at image build time (and then some things
will work, and some things will not work).
https://github.com/oracle/graal/pull/7980
It might not be 100% ready yet for JDK 22.
There also seems to be a difference on how --enable-native-access works (one cannot silence the warning with the switch in native-image).
-J--enable-native-access=ALL-UNNAMED
needs to be passed to native-image
at build time to achieve this silencing.
This issue appears to be stale because it has been open 30 days with no activity. This issue will be closed in 7 days unless Stale
label is removed, a new comment is made, or not-Stale
label is added.
I've tested this with the GraalVM for JDK 22 Community release here (GA released yesterday): https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-22.0.0
Seems to have the same issue:
$ rm -rf META-INF && mkdir -p META-INF/native-image && /disk/graal/builds/graalvm-community-openjdk-22+36.1/bin/java -agentlib:native-image-agent=config-output-dir=META-INF/native-image --enable-native-access=ALL-UNNAMED FFM && /disk/graal/builds/graalvm-community-openjdk-22+36.1/bin/native-image -J--enable-native-access=ALL-UNNAMED --verbose -H:+UnlockExperimentalVMOptions -H:+ForeignAPISupport --no-fallback FFM
Test passed! List sorted: [car, cat, dog, mouse]
Apply file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/jni-config.json
Apply file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/reflect-config.json
Apply file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/resource-config.json
Apply file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/proxy-config.json
Apply file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/serialization-config.json
Apply file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/predefined-classes-config.json
Apply jar:file:///disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/library-support.jar!/META-INF/native-image/com.oracle.svm/thirdparty/native-image.properties
Apply jar:file:///disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/library-support.jar!/META-INF/native-image/com.oracle.svm/polyglot/native-image.properties
Executing [
HOME=/home/sgehwolf \
LANG=en_US.UTF-8 \
PATH=/home/sgehwolf/.local/bin:/home/sgehwolf/bin:/home/sgehwolf/Documents/eclipse/eclipse-2022/eclipse:/home/sgehwolf/.local/bin:/home/sgehwolf/bin:/home/sgehwolf/Documents/eclipse/eclipse-2022/eclipse:/usr/lib64/qt-3.3/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin \
PWD=/home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example \
USE_NATIVE_IMAGE_JAVA_PLATFORM_MODULE_SYSTEM=true \
/disk/graal/builds/graalvm-community-openjdk-22+36.1/bin/java \
-XX:+UseParallelGC \
-XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
-Dtruffle.TrustAllTruffleRuntimeProviders=true \
-Dtruffle.TruffleRuntime=com.oracle.truffle.api.impl.DefaultTruffleRuntime \
-Dgraalvm.ForcePolyglotInvalid=true \
-Dgraalvm.locatorDisabled=true \
-Dsubstratevm.HostLibC=glibc \
--add-exports=java.base/com.sun.crypto.provider=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.access=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.event=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.loader=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.logger=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.misc=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto \
--add-exports=java.base/jdk.internal.module=org.graalvm.nativeimage.base,org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.org.objectweb.asm=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.perf=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.platform=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.ref=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile \
--add-exports=java.base/jdk.internal.reflect=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.util=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.vm.annotation=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal.vm=org.graalvm.nativeimage.builder \
--add-exports=java.base/jdk.internal=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.invoke.util=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.net.www=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.net=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.nio.ch=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile \
--add-exports=java.base/sun.reflect.annotation=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.reflect.generics.factory=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.reflect.generics.reflectiveObjects=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.reflect.generics.repository=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.reflect.generics.scope=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.reflect.generics.tree=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.security.jca=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.security.provider=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.security.ssl=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.security.util=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.security.x509=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.text.spi=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.util.calendar=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.util.cldr=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.util.locale.provider=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.util.locale=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.util.resources=org.graalvm.nativeimage.builder \
--add-exports=java.base/sun.util=org.graalvm.nativeimage.builder \
--add-exports=java.management/com.sun.jmx.mbeanserver=org.graalvm.nativeimage.builder \
--add-exports=java.management/sun.management=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.pointsto \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.aarch64=jdk.graal.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.amd64=jdk.graal.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code.site=jdk.graal.compiler,org.graalvm.nativeimage.builder \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code.stack=jdk.graal.compiler,org.graalvm.nativeimage.builder \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.graal.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto,org.graalvm.truffle.compiler \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.common=jdk.graal.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.pointsto \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.aarch64=jdk.graal.compiler \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.amd64=jdk.graal.compiler \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.riscv64=jdk.graal.compiler \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot=jdk.graal.compiler,org.graalvm.nativeimage.builder \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.meta=jdk.graal.compiler,org.graalvm.nativeimage.base,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto,org.graalvm.truffle.compiler \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.riscv64=jdk.graal.compiler,org.graalvm.nativeimage.builder \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.runtime=jdk.graal.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.pointsto \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.services=jdk.graal.compiler,org.graalvm.nativeimage.builder \
--add-exports=jdk.jfr/jdk.jfr.events=org.graalvm.nativeimage.builder \
--add-exports=jdk.jfr/jdk.jfr.internal.event=org.graalvm.nativeimage.builder \
--add-exports=jdk.jfr/jdk.jfr.internal.jfc=org.graalvm.nativeimage.builder \
--add-exports=jdk.jfr/jdk.jfr.internal=org.graalvm.nativeimage.builder \
--add-exports=jdk.management/com.sun.management.internal=org.graalvm.nativeimage.builder \
-XX:+UseJVMCINativeLibrary \
-Xss10m \
-XX:MaxRAMPercentage=47.64222951120384 \
-XX:GCTimeRatio=9 \
-XX:+ExitOnOutOfMemoryError \
-Djava.awt.headless=true \
'-Dorg.graalvm.vendor=GraalVM Community' \
-Dorg.graalvm.vendorurl=https://www.graalvm.org/ \
'-Dorg.graalvm.vendorversion=GraalVM CE 22+36.1' \
-Dorg.graalvm.version=24.0.0 \
-Dcom.oracle.graalvm.isaot=true \
-Djava.system.class.loader=com.oracle.svm.hosted.NativeImageSystemClassLoader \
-Xshare:off \
-Djdk.reflect.useOldSerializableConstructor=true \
-Djdk.internal.lambda.disableEagerInitialization=true \
-Djdk.internal.lambda.eagerlyInitialize=false \
-Djava.lang.invoke.InnerClassLambdaMetafactory.initializeLambdas=false \
-Djava.lang.invoke.MethodHandle.DONT_INLINE_THRESHOLD=-1 \
-Djava.lang.invoke.MethodHandle.PROFILE_GWT=false \
--enable-native-access=ALL-UNNAMED \
--add-modules=ALL-DEFAULT \
--module-path \
/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/builder/native-image-base.jar:/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/builder/svm-foreign.jar:/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/builder/pointsto.jar:/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/builder/objectfile.jar:/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/builder/svm.jar \
--module \
org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner \
-keepalive \
/proc/93559/comm \
-imagecp \
/home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example \
-imagemp \
/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/library-support.jar \
-H:CLibraryPath=/disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/clibraries/linux-amd64 \
-H:Path@driver=/home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example \
-H:+UnlockExperimentalVMOptions@user \
-H:+ForeignAPISupport@user \
-H:FallbackThreshold@user+api=0 \
-H:-UnlockExperimentalVMOptions@user \
-H:JNIConfigurationResources@file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/jni-config.json+api=META-INF/native-image/jni-config.json \
-H:ReflectionConfigurationResources@file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/reflect-config.json+api=META-INF/native-image/reflect-config.json \
-H:ResourceConfigurationResources@file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/resource-config.json+api=META-INF/native-image/resource-config.json \
-H:DynamicProxyConfigurationResources@file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/proxy-config.json+api=META-INF/native-image/proxy-config.json \
-H:SerializationConfigurationResources@file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/serialization-config.json+api=META-INF/native-image/serialization-config.json \
-H:PredefinedClassesConfigurationResources@file:///home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/META-INF/native-image/predefined-classes-config.json+api=META-INF/native-image/predefined-classes-config.json \
'-H:Class@explicit main-class=FFM' \
'-H:Name@main-class lower case as image name=ffm' \
-H:ImageBuildID@driver=72e4cf08-08a7-37d1-140d-369e254b8717 \
'-H:Features@jar:file:///disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/library-support.jar!/META-INF/native-image/com.oracle.svm/thirdparty/native-image.properties+api=com.oracle.svm.thirdparty.gson.GsonFeature' \
'-H:Features@jar:file:///disk/graal/builds/graalvm-community-openjdk-22+36.1/lib/svm/library-support.jar!/META-INF/native-image/com.oracle.svm/polyglot/native-image.properties+api=com.oracle.svm.polyglot.groovy.GroovyIndyInterfaceFeature,com.oracle.svm.polyglot.scala.ScalaFeature' \
-H:Color@driver=always \
-H:+BuildOutputProgress@driver \
-H:+BuildOutputLinks@driver
]
========================================================================================================================
GraalVM Native Image: Generating 'ffm' (executable)...
========================================================================================================================
[1/8] Initializing... (2.8s @ 0.19GB)
Java version: 22+36, vendor version: GraalVM CE 22+36.1
Graal compiler: optimization level: 2, target machine: x86-64-v3
C compiler: gcc (redhat, x86_64, 13.2.1)
Garbage collector: Serial GC (max heap size: 80% of RAM)
1 user-specific feature(s):
- com.oracle.svm.thirdparty.gson.GsonFeature
------------------------------------------------------------------------------------------------------------------------
1 experimental option(s) unlocked:
- '-H:+ForeignAPISupport' (origin(s): command line)
------------------------------------------------------------------------------------------------------------------------
Build resources:
- 26.49GB of memory (42.3% of 62.55GB system memory, determined at start)
- 12 thread(s) (100.0% of 12 available processor(s), determined at start)
[2/8] Performing analysis... [*****] (10.9s @ 0.47GB)
4,369 reachable types (76.4% of 5,715 total)
4,695 reachable fields (42.9% of 10,950 total)
20,207 reachable methods (49.2% of 41,074 total)
1,413 types, 231 fields, and 959 methods registered for reflection
57 types, 57 fields, and 52 methods registered for JNI access
0 foreign downcalls registered
4 native libraries: dl, pthread, rt, z
[3/8] Building universe... (2.1s @ 0.47GB)
[4/8] Parsing methods... [*] (1.1s @ 0.41GB)
[5/8] Inlining methods... [***] (0.9s @ 0.45GB)
[6/8] Compiling methods... [***] (10.9s @ 0.39GB)
[7/8] Laying out methods... [*] (1.8s @ 0.49GB)
[8/8] Creating image... [*] (1.9s @ 0.56GB)
7.35MB (42.87%) for code area: 11,712 compilation units
9.52MB (55.56%) for image heap: 115,037 objects and 47 resources
276.72kB ( 1.58%) for other data
17.13MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
5.89MB java.base 2.18MB byte[] for code metadata
1012.93kB svm.jar (Native Image) 1.50MB byte[] for java.lang.String
113.51kB java.logging 1.08MB java.lang.String
64.54kB org.graalvm.nativeimage.base 1.02MB java.lang.Class
47.59kB jdk.proxy1 388.11kB heap alignment
45.84kB jdk.proxy3 375.46kB com.oracle.svm.core.hub.DynamicHubCompanion
35.76kB FFM 286.56kB byte[] for general heap data
26.87kB jdk.internal.vm.ci 260.48kB java.util.HashMap$Node
22.07kB org.graalvm.collections 219.66kB java.lang.Object[]
11.42kB jdk.proxy2 219.03kB java.lang.String[]
20.18kB for 4 more packages 2.04MB for 1207 more object types
------------------------------------------------------------------------------------------------------------------------
Recommendations:
HEAP: Set max heap for improved and more predictable memory usage.
CPU: Enable more CPU features with '-march=native' for improved performance.
------------------------------------------------------------------------------------------------------------------------
1.4s (4.0% of total time) in 173 GCs | Peak RSS: 1.00GB | CPU load: 8.93
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/home/sgehwolf/Documents/mandrel/bugs/upstream-mandrel/ffm_jdk22_example/ffm (executable)
========================================================================================================================
Finished generating 'ffm' in 33.6s.
$ ./ffm
Exception in thread "main" com.oracle.svm.core.foreign.ForeignFunctionsRuntime$UnregisteredForeignStubException: Cannot perform downcall with leaf type (long,long,long,long,long)void as it was not registered at compilation time.
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.ForeignFunctionsRuntime.getDowncallStubPointer(ForeignFunctionsRuntime.java:75)
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.NativeEntryPointInfo.makeEntryPoint(NativeEntryPointInfo.java:88)
at java.base@22/jdk.internal.foreign.abi.NativeEntryPoint.make(Target_jdk_internal_foreign_abi_NativeEntryPoint.java:60)
at java.base@22/jdk.internal.foreign.abi.DowncallLinker.getBoundMethodHandle(DowncallLinker.java:87)
at java.base@22/jdk.internal.foreign.abi.x64.sysv.CallArranger.arrangeDowncall(CallArranger.java:132)
at java.base@22/jdk.internal.foreign.abi.x64.sysv.SysVx64Linker.arrangeDowncall(SysVx64Linker.java:62)
at java.base@22/jdk.internal.foreign.abi.AbstractLinker.lambda$downcallHandle0$0(AbstractLinker.java:106)
at java.base@22/jdk.internal.foreign.abi.SoftReferenceCache$Node.get(SoftReferenceCache.java:49)
at java.base@22/jdk.internal.foreign.abi.SoftReferenceCache.get(SoftReferenceCache.java:38)
at java.base@22/jdk.internal.foreign.abi.AbstractLinker.downcallHandle0(AbstractLinker.java:103)
at java.base@22/jdk.internal.foreign.abi.AbstractLinker.downcallHandle(AbstractLinker.java:85)
at FFM.main(FFM.java:37)
at java.base@22/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
So I don't think we can do any more on the Mandrel side.
I've opened https://github.com/oracle/graal/issues/8616 upstream to track this.
I've opened oracle#8616 upstream to track this.
The tracker for this upstream is https://github.com/oracle/graal/issues/8113. Documentation as to what works and what not is at: https://www.graalvm.org/jdk22/reference-manual/native-image/native-code-interoperability/foreign-interface/
Description
With JDK 22, JEP 454 went out of preview. For GraalVM this currently means needing to ship
svm-foreign.jar
in the correct location of the distro. Specifically, next tosvm.jar
in$MANDREL_HOME/lib/svm/builder
. Then, native-image builds pick up the jar and add it to the module path when the generator runs.However, after fixing that, not everything seems to be fully working (might be my test case, but still).
How To Reproduce
Configuration:
Mandrel 24.0 with JDK 22+33.
Expected behavior
Reproducer works the same as when run with
java
:Additional context
Where
svm-foreign.jar
is located changed from 23.1 (JDK 21) to 24.0 (JDK 22). It used to live in ansvm-preview
folder. There also seems to be a difference on how--enable-native-access
works (one cannot silence the warning with the switch in native-image).GraalVM for JDK 22 community (self-built) seems to have a similar issue.