search

raphw / byte-buddy

Runtime code generation for the Java virtual machine.
https://bytebuddy.net
Apache License 2.0
6.28k stars 807 forks source link

Java 22 Support #1609

Closed dwarakaprasad closed 1 month ago

dwarakaprasad commented 7 months ago

Any idea for Java 22 support.

jHipster uses Blockhound for unit testing, which in turn uses byte-buddy. Tests are failing with,

OpenJDK 64-Bit Server VM warning: Option AllowRedefinitionToAddDeleteMethods was deprecated in version 13.0 and will likely be removed in a future release.
WARNING: A Java agent has been loaded dynamically (/tmp/byteBuddyAgent15882293150021270014.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[Byte Buddy] ERROR java.lang.ProcessImpl [null, module java.base, Thread[#1,main,5,main], loaded=true]
java.lang.IllegalArgumentException: Java 22 (66) is not supported by the current version of Byte Buddy which officially supports Java 21 (65) - update Byte Buddy or set reactor.blockhound.shaded.net.bytebuddy.experimental as a VM property
    at reactor.blockhound.shaded.net.bytebuddy.utility.OpenedClassReader.of(OpenedClassReader.java:96)
    at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Default.parse(TypePool.java:879)
    at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Default.doDescribe(TypePool.java:865)
    at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$AbstractBase.describe(TypePool.java:598)
    at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$AbstractBase$Hierarchical.describe(TypePool.java:681)
    at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$DescriptionStrategy$Default$3.apply(AgentBuilder.java:4243)
    at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector.consider(AgentBuilder.java:7956)
    at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy.apply(AgentBuilder.java:5741)
    at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default.doInstall(AgentBuilder.java:11237)
    at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default.installOn(AgentBuilder.java:11155)
    at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$Delegator.installOn(AgentBuilder.java:12927)
    at reactor.blockhound.BlockHound$Builder.instrument(BlockHound.java:581)
    at reactor.blockhound.BlockHound$Builder.install(BlockHound.java:484)
    at reactor.blockhound.BlockHound.install(BlockHound.java:91)
    at reactor.blockhound.junit.platform.BlockHoundTestExecutionListener.<clinit>(BlockHoundTestExecutionListener.java:19)
    at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
    at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
    at java.base/jdk.internal.reflect.MethodHandleAccessorFactory.ensureClassInitialized(MethodHandleAccessorFactory.java:340)
    at java.base/jdk.internal.reflect.MethodHandleAccessorFactory.newConstructorAccessor(MethodHandleAccessorFactory.java:103)
    at java.base/jdk.internal.reflect.ReflectionFactory.newConstructorAccessor(ReflectionFactory.java:173)
    at java.base/java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:549)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
    at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:785)
    at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:725)
    at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1397)
    at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:611)
    at org.junit.platform.launcher.core.LauncherFactory.registerTestExecutionListeners(LauncherFactory.java:195)
    at org.junit.platform.launcher.core.LauncherFactory.createDefaultLauncher(LauncherFactory.java:142)
    at org.junit.platform.launcher.core.LauncherFactory.lambda$openSession$1(LauncherFactory.java:101)
    at org.junit.platform.launcher.core.DefaultLauncherSession.<init>(DefaultLauncherSession.java:53)
    at org.junit.platform.launcher.core.LauncherFactory.openSession(LauncherFactory.java:100)
    at org.junit.platform.launcher.core.LauncherFactory.openSession(LauncherFactory.java:82)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at org.apache.maven.surefire.api.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:125)
    at org.apache.maven.surefire.api.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:62)
    at org.apache.maven.surefire.junitplatform.LazyLauncher.launcher(LazyLauncher.java:68)
    at org.apache.maven.surefire.junitplatform.LazyLauncher.discover(LazyLauncher.java:50)
    at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:52)
    at org.apache.maven.surefire.api.util.DefaultScanResult.applyFilter(DefaultScanResult.java:87)
    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:142)
    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
    at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:[507](https://github.com/jhipster/generator-jhipster/actions/runs/8472839537/job/23215829498?pr=25658#step:17:508))
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)

was able to move forward with the vm argument (reactor.blockhound.shaded.net.bytebuddy.experimental).

Also found a similar issue related to Java 21

raphw commented 7 months ago

Java 22 is supported in the latest version. You would need to ask the library for an update.

dwarakaprasad commented 7 months ago

Java 22 is supported in the latest version. You would need to ask the library for an update.

Awesome thank you!

pderop commented 7 months ago

@raphw ,

Hi Rafael,

I'm currently maintaining BlockHound, and I have upgraded BlockHound 1.0.9 snapshot with latest bytebuddy version (1.14.13). But it seems that there are still problems with JDK22 (only). I managed to do a simple reproducer project that I'm attaching here:

blockhound-issue-410.tgz

For the moment, I cannot say from where the problem is. Can you please give a try to the reproducer ? It's a simple project that is using BlockHound. The junit test verifies that if a blocking task is scheduled in the com.example.NonBlockingThread, then BlockingOperationError exception is thrown.

it works well with all jdk versions, except with jdk 22.

1) Trying with JDK 21:

install JDK 21

build and run tests (should be successful): ./gradlew build -i

try to run the sample application with blockhound agent:

wget https://repo.spring.io/artifactory/snapshot/io/projectreactor/tools/blockhound/1.0.9.BUILD-SNAPSHOT/blockhound-1.0.9.BUILD-SNAPSHOT.jar
java  -javaagent:blockhound-1.0.9.BUILD-SNAPSHOT.jar -XX:+AllowRedefinitionToAddDeleteMethods -jar build/libs/blockhound-example-1.0.0-SNAPSHOT.jar

then the exception is correctly thrown by BlockHound:

Exception in thread "Thread-1" reactor.blockhound.BlockingOperationError: Blocking call! java.lang.Thread.sleep0
    at java.base/java.lang.Thread.sleep0(Thread.java)
    at java.base/java.lang.Thread.sleep(Thread.java:509)
    at com.example.Example.lambda$main$0(Example.java:14)
    at com.example.NonBlockingThread.run(NonBlockingThread.java:11)

2) now trying with JDK 22:

install jdk 22 I'm using :

openjdk version "22" 2024-03-19
OpenJDK Runtime Environment Zulu22.28+91-CA (build 22+36)
OpenJDK 64-Bit Server VM Zulu22.28+91-CA (build 22+36, mixed mode, sharing)

BlockHoundTest > testBlockHound() FAILED java.lang.IllegalStateException: The instrumentation have failed. It looks like you're running on JDK 13+. You need to add '-XX:+AllowRedefinitionToAddDeleteMethods' JVM flag. See https://github.com/reactor/BlockHound/issues/33 for more info. at reactor.blockhound.BlockHound$Builder.testInstrumentation(BlockHound.java:538) at reactor.blockhound.BlockHound$Builder.install(BlockHound.java:501) at reactor.blockhound.BlockHound.install(BlockHound.java:91) at com.example.BlockHoundTest.testBlockHound(BlockHoundTest.java:15)


actually, it's the BlockHound installation that fails, because when it is installing, it tries to verify if instrumentation is working, see [here](https://github.com/reactor/BlockHound/blob/master/agent/src/main/java/reactor/blockhound/BlockHound.java#L526).

(the test is using the special '-XX:+AllowRedefinitionToAddDeleteMethods option, see build.gradle).

- now interestingly, try to execute the application with blockhound agent (it also fails, but this times with some assert jdk errors):

java -javaagent:blockhound-1.0.9.BUILD-SNAPSHOT.jar -XX:+AllowRedefinitionToAddDeleteMethods -jar build/libs/blockhound-example-1.0.0-SNAPSHOT.jar OpenJDK 64-Bit Server VM warning: Option AllowRedefinitionToAddDeleteMethods was deprecated in version 13.0 and will likely be removed in a future release. OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended Exception in thread "main" java.lang.reflect.InvocationTargetException at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:118) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:560) at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:572) Caused by: java.lang.IllegalStateException: The instrumentation have failed. It looks like you're running on JDK 13+. You need to add '-XX:+AllowRedefinitionToAddDeleteMethods' JVM flag. See https://github.com/reactor/BlockHound/issues/33 for more info. at reactor.blockhound.BlockHound$Builder.testInstrumentation(BlockHound.java:538) at reactor.blockhound.BlockHound$Builder.install(BlockHound.java:501) at reactor.blockhound.BlockHound.premain(BlockHound.java:106) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ... 3 more java.lang.instrument ASSERTION FAILED : "!errorOutstanding" with message Outstanding error when calling method in invokeJavaAgentMainMethod at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 627 java.lang.instrument ASSERTION FAILED : "success" with message invokeJavaAgentMainMethod failed at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 466 java.lang.instrument ASSERTION FAILED : "result" with message agent load/premain call failed at src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 429 FATAL ERROR in native method: processing of -javaagent failed, processJavaStart failed Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.dylib+0x52cc44] jni_FatalError+0x7c V [libjvm.dylib+0x699990] JvmtiExport::post_vm_initialized()+0x2b8 V [libjvm.dylib+0xa20a08] Threads::create_vm(JavaVMInitArgs, bool)+0x6c4 V [libjvm.dylib+0x54b69c] JNI_CreateJavaVM+0x74 C [libjli.dylib+0xa510] JavaMain+0x100 C [libjli.dylib+0xd5e0] ThreadJavaMain+0xc C [libsystem_pthread.dylib+0x6fa8] _pthread_start+0x94 Abort trap: 6



I'm investigating, but that would be nice if you could play with the reproducer , maybe you will figure out somethikng ? (the assertions errors from the jdk are very strange ?)

thanks !
raphw commented 7 months ago

Quite honestly, this looks like a JVM bug to me that is related to AllowRedefinitionToAddDeleteMethods. On the long run you need to find a way around this option as it will be removed anyways, so I might be looking into that while also reporting the bug to OpenJDK.

pderop commented 7 months ago

Thank you Rafael;

in fact, I tend to do agree, because I managed to hack BlockHound in order to instrument only one single non-native method, and in this case, it works well with JDK 22. Now, looking into JDK 22, the AllowRedefinitionToAddDeleteMethods seems to be still there, so it sounds like a OpenJDK 22 bug, I will manage to isolate the issue and submit them a bug. Then, for the long term, I'll start to investigate another way to instrument native methods (I have no ideas for the moment).

Thank you.