search

RuedigerMoeller / fast-serialization

FST: fast java serialization drop in-replacement
Apache License 2.0
1.59k stars 247 forks source link

StackOverflowError while reading serialized TreeSet #165

Closed IhorProkopov closed 7 years ago

IhorProkopov commented 7 years ago

Hi! Very ofter cought this error when trying read file with TreeSet.

Caused by: java.lang.StackOverflowError at org.nustaq.serialization.util.FSTInt2ObjectMap.putNext(FSTInt2ObjectMap.java:85) at org.nustaq.serialization.util.FSTInt2ObjectMap.putHash(FSTInt2ObjectMap.java:76) at org.nustaq.serialization.util.FSTInt2ObjectMap.putNext(FSTInt2ObjectMap.java:85) at org.nustaq.serialization.util.FSTInt2ObjectMap.putHash(FSTInt2ObjectMap.java:76)

Here is how I serialize it:

try (FileOutputStream fos = new FileOutputStream(fileName); BufferedOutputStream out = new BufferedOutputStream(fos); GzipCompressorOutputStream gos = new GzipCompressorOutputStream(out, gzipParameters)) { conf.get().encodeToStream(gos, documentData); }

And desirealize:

try (FileInputStream fin = new FileInputStream(fileName); in = new BufferedInputStream(new ByteArrayInputStream(IOUtils.toByteArray(fin))); GzipCompressorInputStream gis = new GzipCompressorInputStream(in)) { return (TreeSet<IndexDocumentData>) conf.get().decodeFromStream(gis); }

But when trying do reproduce this bug locally everyting is ok.

Thanks

RuedigerMoeller commented 7 years ago

by default TreeSet uses default JDK serialization methods which is quite stack heavy, try registering a custom serializer fro TreeSet

FSTConfiguration conf = FSTConfiguration.createDefaultConfiguration();
conf.registerSerializer(TreeSet.class, new FSTCollectionSerializer, false );

if this fails (construction issues), you need to roll your own Sereializer (basically copying FSTCollectionSerializer, but hardcode constructor calls to TreeSet).