apache / fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
https://fury.apache.org/
Apache License 2.0
3.13k stars 252 forks source link

IllegalAccessError thrown from JIT serializer with async compilation #1879

Closed theigl closed 1 month ago

theigl commented 1 month ago

Search before asking

Version

0.8.0

Component(s)

Java

Minimal reproduce step

I'm trying to serialize the static inner class DelegatedPage from SessionQotaManagingDataStore that is used in Apache Wicket.

Fury can serialize the class just fine but as soon as async code generation kicks in, an IllegalAccessError is thrown.

The following test demonstrates the problem. It does not fail, but it logs the error to the console. If withAsyncCompilation is set to false, the error goes away.

package com.synexit.base.web.util.serializer.repro;

import org.apache.fury.Fury;
import org.apache.fury.ThreadSafeFury;
import org.apache.fury.config.Language;
import org.junit.jupiter.api.Test;

import java.io.Serializable;
import java.util.concurrent.ConcurrentLinkedQueue;

import static org.junit.jupiter.api.Assertions.assertNotNull;

class JitTest {

    @Test
    void shouldSerializeWithAsyncCompilation() throws InterruptedException {
        final ThreadSafeFury fury = Fury.builder()
                .withLanguage(Language.JAVA)
                .requireClassRegistration(false)
                .withRefTracking(true)
                .withAsyncCompilation(true)
                .buildThreadSafeFuryPool(1, 1);

        final Store store = new Store();
        store.addPage(new Store.SerializedPage(1));

        final Object v = store.getData();
        for (int i = 0; i < 1000; i++) {
            final byte[] d = fury.serialize(v);
            final Object deserialized = fury.deserialize(d);
            assertNotNull(deserialized);
        }
        Thread.sleep(1000);
        for (int i = 0; i < 1000; i++) {
            final byte[] d = fury.serialize(v);
            final Object deserialized = fury.deserialize(d);
            assertNotNull(deserialized);
        }
    }

    public static class Store {

        public static final class SerializedPage {

            private final int pageId;
            public SerializedPage(int pageId) {
                this.pageId = pageId;
            }

            public int getPageId() {
                return pageId;
            }
        }

        private final CountLimitedData data;

        public Store() {
            this.data = new CountLimitedData(1);
        }

        public void addPage(SerializedPage page) {
            this.data.addPage(page);
        }

        public Object getData() {
            return data;
        }

        abstract static class SessionData implements Serializable {
            final ConcurrentLinkedQueue<DelegatedPage> pages = new ConcurrentLinkedQueue();

            SessionData() {
            }

            synchronized void addPage(SerializedPage page) {
                this.pages.add(new DelegatedPage(page.getPageId(), 0));
            }
        }

        static class DelegatedPage implements Serializable {
            public final int pageId;
            public final long pageSize;

            public DelegatedPage(int pageId, long pageSize) {
                this.pageId = pageId;
                this.pageSize = pageSize;
            }
        }

        static class CountLimitedData extends SessionData {
            private final int maxPages;

            public CountLimitedData(int maxPages) {
                this.maxPages = maxPages;
            }
        }
    }
}

What did you expect to see?

Classes that can be serialized with code generation should be serializable with async code generation as well.

What did you see instead?

java.lang.IllegalAccessError: failed to access class org.wicketstuff.datastores.common.SessionQuotaManagingDataStore$DelegatedPage from class org.wicketstuff.datastores.common.SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0 (org.wicketstuff.datastores.common.SessionQuotaManagingDataStore$DelegatedPage is in unnamed module of loader 'app'; org.wicketstuff.datastores.common.SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0 is in unnamed module of loader org.apache.fury.util.ClassLoaderUtils$ByteArrayClassLoader @31123c7a)
    at org.wicketstuff.datastores.common.SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0.<init>(SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0.java:59)
    at org.apache.fury.serializer.Serializers.createSerializer(Serializers.java:129)
    at org.apache.fury.serializer.Serializers.newSerializer(Serializers.java:104)
    at org.apache.fury.resolver.ClassResolver$1.onSuccess(ClassResolver.java:941)
    at org.apache.fury.resolver.ClassResolver$1.onSuccess(ClassResolver.java:938)
    at org.apache.fury.builder.JITContext.lambda$registerSerializerJITCallback$0(JITContext.java:94)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Exception in thread "fury-jit-compiler-2" java.lang.IllegalAccessError: failed to access class org.wicketstuff.datastores.common.SessionQuotaManagingDataStore$DelegatedPage from class org.wicketstuff.datastores.common.SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0 (org.wicketstuff.datastores.common.SessionQuotaManagingDataStore$DelegatedPage is in unnamed module of loader 'app'; org.wicketstuff.datastores.common.SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0 is in unnamed module of loader org.apache.fury.util.ClassLoaderUtils$ByteArrayClassLoader @31123c7a)
    at org.wicketstuff.datastores.common.SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0.<init>(SessionQuotaManagingDataStore_SizeLimitedDataFuryRefCodec_0.java:59)
    at org.apache.fury.serializer.Serializers.createSerializer(Serializers.java:129)
    at org.apache.fury.serializer.Serializers.newSerializer(Serializers.java:104)
    at org.apache.fury.resolver.ClassResolver$1.onSuccess(ClassResolver.java:941)
    at org.apache.fury.resolver.ClassResolver$1.onSuccess(ClassResolver.java:938)
    at org.apache.fury.builder.JITContext.lambda$registerSerializerJITCallback$0(JITContext.java:94)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)

Anything Else?

No response

Are you willing to submit a PR?

chaokunyang commented 1 month ago

Hi @theigl , thanks for reporting this bug. This is not easy to capture since it doesn't affect the execution but only performance. I fixed it in https://github.com/apache/fury/pull/1883