apache / fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
https://fury.apache.org/
Apache License 2.0
3.12k stars 248 forks source link

[Question] <title>What does java.lang.RuntimeException: Deserialize failed, type of read objects are: [... #1953

Open cquezel opened 4 days ago

cquezel commented 4 days ago

Question

what does :

java.lang.RuntimeException: Deserialize failed, type of read objects are: [ ... class java.lang.String, null]

mean? How do I diagnose this situation? The ... in the error message is a list of class names. In this case, there are ~7500 classes many of which are repeated. The last class name in the message is null.

Could this be a stackoverflow? When I do this with java serialisation, I increase the stack to 33MB which is the value I used in this test.

The code I use is:

    private static Fury fury;

    private static Fury getFury() {
        if (fury == null) {
            fury = Fury.builder().withLanguage(Language.JAVA).requireClassRegistration(false).withRefTracking(true).build();
        }
        return fury; 
    }

    private static <T> T writeFury(FileInputStream fis, Class<T> klass) {
        return getFury().deserializeJavaObject(new FuryInputStream(fis), klass);
    }

    private static void readFury(FileOutputStream fos, Object objet) {
        getFury().serializeJavaObject(fos, objet);
    }
chaokunyang commented 4 days ago

You should use ThreadSafeFury as a static variable. If it still fails, we need the reproducible code to find out what happend.

cquezel commented 4 days ago

The code in single threaded. Would this make a difference?

chaokunyang commented 4 days ago

Nope, please provide the reproducible code

cquezel commented 3 days ago

I think I have identified the problem. Will try to build a small test case next week. It seams related to the order of readObject calls. I think this is similar to https://github.com/apache/fury/issues/1455

cquezel commented 3 days ago

BTW, Although deserialization did not work, Fury serialization took 21 seconds while Java took 3 minutes (12% 21/180)

chaokunyang commented 3 days ago

BTW, Although deserialization did not work, Fury serialization took 21 seconds while Java took 3 minutes (12% 21/180)

Do you have any logs for this. I guess it's caused by Fury JIT. The fury JIT takes time, for the first serialization of every type, it will be slower, but later it will be much faster. And fury also supports async compilation, you could enable it by FuryBuilder#withAsyncCompilation(true)

cquezel commented 3 days ago

I see no problems with Fury being 8.5 times faster than JDK serialisation. Were you expecting better performance?

cquezel commented 3 days ago

adding withAsyncCompilation(true) did not change performances.

chaokunyang commented 3 days ago

Please provide reproduction code, it depends on your objects. Some types may define their serialization method, and it may be not as fast as fury, which may slow the whole serialization.