ClassLoader issues reading proxy with ThreadPoolFury

[theigl]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Version

0.8.0

Component(s)

Java

Minimal reproduce step

It is quite difficult to simulate this behavior:

import org.apache.fury.Fury;
import org.apache.fury.ThreadSafeFury;
import org.apache.fury.config.Language;
import org.junit.jupiter.api.Test;

import java.io.Serializable;
import java.lang.reflect.*;
import java.util.concurrent.CountDownLatch;

import static org.junit.jupiter.api.Assertions.assertEquals;

class ProxySerializerTest {

    static class MyClassLoader extends ClassLoader {
        //dummy implementation
    }

    @Test
    void readProxy() throws InterruptedException, ClassNotFoundException {
        final MyClassLoader myClassLoader = new MyClassLoader();
        final Class<?> bf = Class.forName(getClass().getPackageName() + ".TestClass$TestInterface", true, myClassLoader);
        final Class<?>[] interfaces = new Class<?>[] {bf, Serializable.class};
        final Object type = Proxy.newProxyInstance(myClassLoader, interfaces, (proxy, method, args) -> null);

        final ThreadSafeFury fury = Fury.builder()
                .withClassLoader(myClassLoader)
                .withLanguage(Language.JAVA)
                .requireClassRegistration(false)
                .buildThreadSafeFuryPool(0, 10);

        final CountDownLatch latch = new CountDownLatch(1);
        new Thread(() -> {
            final byte[] s = fury.serialize(type);
            assertEquals(fury.deserialize(s), type);
            latch.countDown();
        }).start();
        latch.await();
    }
}

In the same package, put the following class:

public class TestClass {

    interface TestInterface {

    }
}

What did you expect to see?

I can read proxies for private interfaces loaded during application startup.

What did you see instead?

The following exception is thrown in a Spring Boot app when reading a proxy created by Spring's SerializableTypeWrapper:

Caused by: java.lang.IllegalArgumentException: non-public interface is not defined by the given loader
    at java.base/java.lang.reflect.Proxy$ProxyBuilder.proxyClassContext(Proxy.java:812)
    at java.base/java.lang.reflect.Proxy$ProxyBuilder.<init>(Proxy.java:638)
    at java.base/java.lang.reflect.Proxy.lambda$getProxyConstructor$1(Proxy.java:440)
    at java.base/jdk.internal.loader.AbstractClassLoaderValue$Memoizer.get(AbstractClassLoaderValue.java:329)
    at java.base/jdk.internal.loader.AbstractClassLoaderValue.computeIfAbsent(AbstractClassLoaderValue.java:205)
    at java.base/java.lang.reflect.Proxy.getProxyConstructor(Proxy.java:438)
    at java.base/java.lang.reflect.Proxy.newProxyInstance(Proxy.java:1034)
    at org.apache.fury.serializer.JdkProxySerializer.read(JdkProxySerializer.java:83)
    at org.apache.fury.Fury.readDataInternal(Fury.java:958)
    at org.apache.fury.Fury.readRef(Fury.java:873)
    at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:368)
    at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:312)
    at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:246)
    at org.apache.fury.serializer.ReplaceResolveSerializer.readObject(ReplaceResolveSerializer.java:316)
    at org.apache.fury.serializer.ReplaceResolveSerializer.read(ReplaceResolveSerializer.java:305)
    at org.apache.wicket.spring.SpringBeanLocatorFuryRefCodec_0.readFields1$(SpringBeanLocatorFuryRefCodec_0.java:159)
    at org.apache.wicket.spring.SpringBeanLocatorFuryRefCodec_0.read(SpringBeanLocatorFuryRefCodec_0.java:192)

The class was loaded by Spring during application startup using the AppClassLoader. ThreadPoolFury always uses the current thread context which is TomcatEmbeddedClassLoader in my case. It is not possible to force ThreadPoolFury to use the class loader provided to Fury.

Anything Else?

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[chaokunyang]

Could you try to invoke ThreadSafeFury#setClassLoader? It should work

[theigl]

I will try this in my application tomorrow! It does not work in my test, but I just realized that my test is flawed.

[theigl]

Unfortunately, it does not work. Here is a simplified test-case that demonstrates the issue:

import org.apache.fury.Fury;
import org.apache.fury.ThreadSafeFury;
import org.apache.fury.config.Language;
import org.junit.jupiter.api.Test;

import java.util.concurrent.CountDownLatch;

import static org.junit.jupiter.api.Assertions.assertInstanceOf;

class ClassLoaderTest {

    static class MyClassLoader extends ClassLoader {}

    @Test
    void shouldUseProvidedClassLoader() throws InterruptedException {
        final MyClassLoader myClassLoader = new MyClassLoader();
        final ThreadSafeFury fury = Fury.builder()
                .withClassLoader(myClassLoader)
                .withLanguage(Language.JAVA)
                .requireClassRegistration(false)
                .buildThreadSafeFuryPool(1, 1);
        fury.setClassLoader(myClassLoader);

        final CountDownLatch latch = new CountDownLatch(1);
        new Thread(() -> {
            final ClassLoader t = fury.getClassLoader();
            assertInstanceOf(MyClassLoader.class, t);
            latch.countDown();
        }).start();
        latch.await();
    }
}

[chaokunyang]

It's not how this API be used, you should invoke fury.setClassLoader(myClassLoader); just before you invoke Fury#deserialize. This set classloader is cached as a thread local variable, thus is only visible on your setting thread

[theigl]

Ah OK! Thanks for clarifying this.

Is there a reason the API has to be so complicated for ThreadPoolFury? I understand that it has to be this way if I want to customize the class loader on a per-call or per-thread basis. But in my case I just want to use a "hard-coded" class loader for each and every thread. Could it not use the class loader passed to .withClassLoader(xyz) by default?

[chaokunyang]

You can do that by:

new ThreadPoolFury(classloader -> Fury.builder()
                .withClassLoader(myClassLoader)
                .withLanguage(Language.JAVA)
                .requireClassRegistration(false))

[chaokunyang]

But this is indeed a bug, we need to use classloader set in FuryBuilder for ThreadSafeFury too. Would you like to submit a PR?

[theigl]

I'm on holiday for the next 7-10 days. If the issue is still open when I come back, I can take a look!