Empty ClassNotFoundException when using Unsafe serializer

[mattwigway]

I'm seeing roughly the same stack trace that was reported in #48, but only when I use FSTConfiguration.createUnsafeBinaryConfiguration(): Exception in thread "main" java.io.IOException: java.lang.RuntimeException: class not found CLASSNAME: loader:sun.misc.Launcher$AppClassLoader@14dad5dc (full stack trace at end of ticket). I'm using pretty simple code:

        TransportNetwork network = TransportNetwork.fromDirectory(new File(args[0]));

        FSTConfiguration config = FSTConfiguration.createUnsafeBinaryConfiguration();

        File netfile = File.createTempFile("network", ".dat");
        OutputStream os = new FileOutputStream(netfile);
        FSTObjectOutput out = new FSTObjectOutput(os, config);
        out.writeObject(network);
        out.close();
        os.close();

        InputStream is = new FileInputStream(netfile);
        FSTObjectInput in = new FSTObjectInput(is, config);
        in.readObject();
        in.close();
        is.close();

TransportNetwork is a fairly large class that contains the street and transit network of a city (Portland, Oregon, USA in my test case) represented as fields. The graph is pretty simple; there are minimal cycles and most data is stored in large primitive arrays, which is why I want to use Unsafe serialization.

The class is defined here.

If I replace FSTConfiguration.createUnsafeBinaryConfiguration(); with FSTConfiguration.createDefaultConfiguration(); everything works.

Exception in thread "main" java.io.IOException: java.io.IOException: java.lang.RuntimeException: class not found CLASSNAME: loader:sun.misc.Launcher$AppClassLoader@14dad5dc
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:243)
    at com.conveyal.r5.analyst.cluster.lambda.LambdaWorker.main(LambdaWorker.java:132)
Caused by: java.io.IOException: java.lang.RuntimeException: class not found CLASSNAME: loader:sun.misc.Launcher$AppClassLoader@14dad5dc
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:243)
    at gnu.trove.map.hash.TIntObjectHashMap.readExternal(TIntObjectHashMap.java:1002)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:544)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:327)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:307)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:241)
    ... 1 more
Caused by: java.lang.RuntimeException: class not found CLASSNAME: loader:sun.misc.Launcher$AppClassLoader@14dad5dc
    at org.nustaq.serialization.FSTClazzNameRegistry.classForName(FSTClazzNameRegistry.java:235)
    at org.nustaq.serialization.FSTClazzNameRegistry.classForName(FSTClazzNameRegistry.java:190)
    at org.nustaq.serialization.FSTClazzNameRegistry.decodeClass(FSTClazzNameRegistry.java:173)
    at org.nustaq.serialization.coders.FSTBytezDecoder.readClass(FSTBytezDecoder.java:374)
    at org.nustaq.serialization.FSTObjectInput.readClass(FSTObjectInput.java:933)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:343)
    at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:327)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:307)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:241)
    ... 19 more
Caused by: java.lang.ClassNotFoundException: 
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.nustaq.serialization.FSTClazzNameRegistry.classForName(FSTClazzNameRegistry.java:197)
    ... 27 more

[mattwigway]

I had initially pasted the wrong stack trace. Now fixed.

I notice that the problem appears to be in TIntObjectHashMap.readExternal, which is from the Trove project.

[mattwigway]

TIntObjectHashMap is a generic (i.e. you make a TIntObjectHashMap), and the line where it is failing is a line where it's reading an instance of the generic class.

[mattwigway]

It's reading a TIntObjectHashMap, so the class it's having trouble with is TIntList, which is itself externalizable. I suspect that the serializer is not properly writing the class info when an Externalizable object is in a generic.

[mattwigway]

And TIntList is an interface as well, it's actually reading a TIntArrayList.

[mattwigway]

If I preregister class TIntArrayList in my config I get:

Exception in thread "main" java.io.IOException: java.io.IOException: java.lang.RuntimeException: unable to find class for code 121
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:243)
    at com.conveyal.r5.analyst.cluster.lambda.LambdaWorker.main(LambdaWorker.java:134)
Caused by: java.io.IOException: java.lang.RuntimeException: unable to find class for code 121
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:243)
    at gnu.trove.map.hash.TIntObjectHashMap.readExternal(TIntObjectHashMap.java:1002)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:544)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:708)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:562)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:370)
    at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:327)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:307)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:241)
    ... 1 more
Caused by: java.lang.RuntimeException: unable to find class for code 121
    at org.nustaq.serialization.FSTClazzNameRegistry.decodeClass(FSTClazzNameRegistry.java:180)
    at org.nustaq.serialization.coders.FSTBytezDecoder.readClass(FSTBytezDecoder.java:374)
    at org.nustaq.serialization.FSTObjectInput.readClass(FSTObjectInput.java:933)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:343)
    at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:327)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:307)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:241)
    ... 19 more```

[RuedigerMoeller]

hm, actually UnsafeSerializer should behave identical to defaultSeerializer .. Do you serialize and deserialize with same serializer type ? Serializing with defaultSerializer and trying to read with UnsafeSerializer will not work (UnsafeSerializer uses native byte order).

[mattwigway]

Yes, I'm using the same configuration for both.

On Fri, May 20, 2016 at 4:13 PM, moru0011 notifications@github.com wrote:

hm, actually UnsafeSerializer schould behave identical to defaultSeerializer .. Do you serialize and deserialize with same serializer type ? Serializing with defaultSerializer and trying to read with UnsafeSerializer probably might not work (UnsafeSerializer uses native byte order).

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/RuedigerMoeller/fast-serialization/issues/130#issuecomment-220707312

Matthew Wigginton Conway [WIG-in-tun CON-way] Transportation Analytics/Open Source Washington, DC

[RuedigerMoeller]

ok, I'll flag this as bug. Minor because you can use defaultSerializer meanwhile. Frequently the performancedifference is neglectible. As I am very busy currently it might take some time until i can provide a fix

[mattwigway]

Thanks! Let me know if there's anything else I can provide.