[Java] In an environment where the ThreadContextClassLoader is not an AppClassLoader, deserialization during asynchronous execution in ForkJoinPool#commonPool may result in a ClassNotFoundException error. #1884
[X] I had searched in the issues and found no similar issues.
Version
Fury 0.8.0
JDK openjdk version "17.0.4"
Component(s)
Java
Minimal reproduce step
package com.alioth.bootstrap;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
/**
* Loading classes from external locations
* similar to the behavior of Spring Boot's LaunchedURLClassLoader
*
*/
public class ExternalClassLoader extends ClassLoader {
/**
* Classes to be loaded externally
*/
public static final String[] externalClass = {
"com.alioth.bootstrap.ExternalClassBootStrap",
"com.alioth.bootstrap.ExternalClass"
};
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
for (String clz : externalClass) {
if (clz.equals(name)) {
Class<?> c = findLoadedClass(name);
if (c == null) {
try {
int index = name.lastIndexOf('.');
String pos = name.substring(index + 1) + ".class";
// prefix System.getProperty("user.dir")
Path path = Paths.get(pos);
byte[] bytes = Files.readAllBytes(path);
return defineClass(name, bytes, 0, bytes.length);
} catch (Exception ignore) {
}
}
}
}
return super.findClass(name);
}
}
/**
* Classes used for Fury testing
* Should not appear in the default ClassPath.
*/
public class ExternalClass {
private int a;
private String b;
public int getA() {
return a;
}
public void setA(int a) {
this.a = a;
}
public String getB() {
return b;
}
public void setB(String b) {
this.b = b;
}
}
package com.alioth.bootstrap;
import org.apache.fury.Fury;
import org.apache.fury.ThreadLocalFury;
import org.apache.fury.config.Language;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.ForkJoinTask;
/**
* The class used to launch tests
* Should not appear in the default ClassPath.
*/
public class ExternalClassBootStrap {
public static final ClassLoader loader = ExternalClassBootStrap.class.getClassLoader();
public static final ThreadLocalFury FURY = Fury.builder()
.withLanguage(Language.JAVA)
.requireClassRegistration(false)
.withClassLoader(loader)
.buildThreadLocalFury();
static byte[] bytes;
static {
ExternalClass obj = new ExternalClass();
obj.setA(1);
obj.setB("2");
bytes = FURY.serialize(obj);
}
public static void main(String[] args) throws Exception {
// The withClassLoader method of FuryBuilder only affects the Fury instance in the current thread, and does not affect Fury instances in other threads.
// Other Fury instances will, by default, use the ThreadContextClassLoader.
System.out.println(Thread.currentThread() + " Classloader " + loader);
Object o1 = FURY.deserialize(bytes);
ForkJoinTask<?> task = ForkJoinPool.commonPool().submit(() -> {
ClassLoader cl = Thread.currentThread().getContextClassLoader();
System.out.println(Thread.currentThread() + " ClassLoader " + cl);
// ClassNotFoundException occurred because the ThreadContextClassLoader was used instead of the specified loader.
Object o2 = FURY.deserialize(bytes);
});
// Wait for a moment to prevent the ForkJoinPool tasks from being executed by the main thread instead of asynchronously.
Thread.sleep(500);
task.get();
}
}
package com.alioth.bootstrap;
import org.junit.jupiter.api.Test;
import java.lang.reflect.Method;
import java.util.concurrent.ForkJoinPool;
public class FuryTest {
@Test
public void testFuryForkJoinPoolAndClassLoader() throws Exception {
ClassLoader loader = new ExternalClassLoader();
Thread.currentThread().setContextClassLoader(loader);
String bootstrap = "com.alioth.bootstrap.ExternalClassBootStrap";
Class<?> clz = Class.forName(bootstrap, true, loader);
Method main = clz.getMethod("main", String[].class);
main.invoke(null, new Object[]{new String[]{}});
}
}
What did you expect to see?
Complete the deserialization process correctly.
What did you see instead?
An incorrect class loader was selected, making it unable to retrieve the metadata correctly
The stack trace is as follows:
Thread[main,5,main] Classloader com.alioth.bootstrap.ExternalClassLoader@69b2283a
Thread[ForkJoinPool.commonPool-worker-1,5,main] ClassLoader jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b
2024-10-13 12:02:38 INFO Fury:160 [ForkJoinPool.commonPool-worker-1] - Created new fury org.apache.fury.Fury@40f1c094
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: java.lang.IllegalStateException: Class com.alioth.bootstrap.ExternalClass not found from classloaders [jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b, jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b]
at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(ForkJoinTask.java:605)
at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:981)
at com.alioth.bootstrap.ExternalClassBootStrap.main(ExternalClassBootStrap.java:45)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.alioth.bootstrap.FuryTest.testMultiThreadClassLoader(FuryTest.java:25)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
Caused by: java.lang.IllegalStateException: java.lang.IllegalStateException: Class com.alioth.bootstrap.ExternalClass not found from classloaders [jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b, jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b]
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(ForkJoinTask.java:604)
... 76 more
Caused by: java.lang.IllegalStateException: Class com.alioth.bootstrap.ExternalClass not found from classloaders [jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b, jdk.internal.loader.ClassLoaders$AppClassLoader@63947c6b]
at org.apache.fury.resolver.ClassResolver.loadClass(ClassResolver.java:1849)
at org.apache.fury.resolver.ClassResolver.loadClass(ClassResolver.java:1828)
at org.apache.fury.resolver.ClassResolver.populateBytesToClassInfo(ClassResolver.java:1765)
at org.apache.fury.resolver.ClassResolver.loadBytesToClassInfo(ClassResolver.java:1750)
at org.apache.fury.resolver.ClassResolver.readClassInfoFromBytes(ClassResolver.java:1737)
at org.apache.fury.resolver.ClassResolver.readClassInfo(ClassResolver.java:1659)
at org.apache.fury.Fury.readRef(Fury.java:860)
at org.apache.fury.Fury.deserialize(Fury.java:792)
at org.apache.fury.Fury.deserialize(Fury.java:714)
at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:127)
at com.alioth.bootstrap.ExternalClassBootStrap.lambda$main$0(ExternalClassBootStrap.java:40)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1375)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: java.lang.ClassNotFoundException: com.alioth.bootstrap.ExternalClass
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at org.apache.fury.resolver.ClassResolver.loadClass(ClassResolver.java:1838)
... 16 more
Anything Else?
The ForkJoinWorkerThread in ForkJoinPool#commonPool sets the ThreadContextClassLoader in LTS JDK 11 JDK17 JDK21 ,however, this issue does not occur in JDK 8., which can cause the aforementioned issues for some common APIs, such as CompletableFuture when no specific thread pool is provided.
ForkJoinWorkerThread(ThreadGroup group, ForkJoinPool pool,
boolean useSystemClassLoader, boolean isInnocuous) {
super(group, null, pool.nextWorkerThreadName(), 0L);
... // Omitting certain content.
if (useSystemClassLoader) // true
super.setContextClassLoader(ClassLoader.getSystemClassLoader()); // AppClassLoader
}
2.Why does the FuryBuilder#withClassLoader method's ClassLoader only apply to the Fury instance in the current thread, and not to all Fury instances under the ThreadSafeFury? Is this due to considerations related to GC?
Search before asking
Version
Fury 0.8.0 JDK openjdk version "17.0.4"
Component(s)
Java
Minimal reproduce step
What did you expect to see?
Complete the deserialization process correctly.
What did you see instead?
An incorrect class loader was selected, making it unable to retrieve the metadata correctly
The stack trace is as follows:
Anything Else?
2.Why does the FuryBuilder#withClassLoader method's ClassLoader only apply to the Fury instance in the current thread, and not to all Fury instances under the ThreadSafeFury? Is this due to considerations related to GC?
Are you willing to submit a PR?