apache / fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
https://fury.apache.org/
Apache License 2.0
2.97k stars 218 forks source link

Fury eagerly loads non-base module classes #1695

Open mtf90 opened 2 months ago

mtf90 commented 2 months ago

Search before asking

Version

Fury 0.5.1, as released on Maven Central

mvn --version
Apache Maven 3.9.7 (8b094c9513efc1b9ce2d952b3b9c8eaedaf8cbf0)
Maven home: /usr/share/java/maven
Java version: 11.0.23, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-11-openjdk
Default locale: de_DE, platform encoding: UTF-8
OS name: "linux", version: "6.9.5-arch1-1", arch: "amd64", family: "unix"

Component(s)

Java

Minimal reproduce step

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.github.mtf90</groupId>
    <artifactId>fury-module</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.release>11</maven.compiler.release>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.fury</groupId>
            <artifactId>fury-core</artifactId>
            <version>0.5.1</version>
        </dependency>
    </dependencies>
</project>

module-info.java:

module com.github.mtf90.fury {
    requires org.apache.fury.core;
}

Application:

package com.github.mtf90;

import org.apache.fury.Fury;

public class App {

    public static void main(String[] args) {
        Fury fury = Fury.builder().requireClassRegistration(false).build();
        fury.serialize(new MyObject());
    }

    static class MyObject {}

}

What did you expect to see?

The programm should run without any issues.

What did you see instead?

java.lang.NoClassDefFoundError: java/sql/Date
    at org.apache.fury.core@0.5.1/org.apache.fury.type.TypeUtils.<clinit>(TypeUtils.java:86)
    at org.apache.fury.core@0.5.1/org.apache.fury.resolver.ClassResolver.<init>(ClassResolver.java:265)
    at org.apache.fury.core@0.5.1/org.apache.fury.Fury.<init>(Fury.java:134)
    at org.apache.fury.core@0.5.1/org.apache.fury.config.FuryBuilder.newFury(FuryBuilder.java:332)
    at org.apache.fury.core@0.5.1/org.apache.fury.config.FuryBuilder.build(FuryBuilder.java:347)
    at com.github.mtf90.fury/com.github.mtf90.App.main(App.java:8)
Caused by: java.lang.ClassNotFoundException: java.sql.Date
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
    ... 6 more
Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.apache.fury.core@0.5.1/org.apache.fury.memory.Platform.<clinit>(Platform.java:34)
    at org.apache.fury.core@0.5.1/org.apache.fury.config.FuryBuilder.newFury(FuryBuilder.java:336)
    at org.apache.fury.core@0.5.1/org.apache.fury.config.FuryBuilder.build(FuryBuilder.java:347)
    at com.github.mtf90.fury/com.github.mtf90.App.main(App.java:8)
Caused by: java.lang.UnsupportedOperationException: Unsafe is not supported in this platform.
    at org.apache.fury.core@0.5.1/org.apache.fury.util.unsafe._JDKAccess.<clinit>(_JDKAccess.java:76)
    ... 4 more

Anything Else?

As far as I can tell, this is issue comes down to the static initializations in the TypeUtils class which load the classes java.sql.Date and java.sql.Timestamp. You can workaround this issue by adding

requires java.sql;
requires jdk.unsupported;

to the module-info.java. However, I would argue that Fury should not force users to add (even unsupported) dependencies to their project that are not actually needed. I'm not sure about the direct dependencies here (i.e., whether the Unsafe stuff is only needed as a fallback since it doesn't find Date in the first place).

Given that the types are initialized via TypeRef.of(Date.class) and TypeDef.of(Timestamp.class) (i.e., there does not happen any major background magic) the initializations may simply be removed as they can be easily added in user-land if needed?

Alternatively, they could be added on-demand, e.g., only if user-land actually uses these types. Although this could be a little bit complicated given the static nature of the initialization.

A third idea may be to look at java.util.Date as potential substitute for at least java.sql.Date.

Edit: A fourth idea would be to actually declare java.sql and jdk.unsupported as transitive dependencies of the fury.core module. However, this would require Fury to support full JPMS modules and not just Automatic-Module-Names (cf. #1341).

Are you willing to submit a PR?

chaokunyang commented 2 months ago

Lazy on-demand loading is better, we may be maintain a classname to class Id and serializer map in ClassResolver. This is also useful if some class doesn't existin some version but we need put a placeholder for class id in class resolver