davidmoten
commented
8 years ago Just FlatBuffers serialization is supported at the moment but I might need to add a Kryo option given that 2GB limit. I'll have a think about it.
Open ksaurab opened 8 years ago
davidmoten
commented
8 years ago Just FlatBuffers serialization is supported at the moment but I might need to add a Kryo option given that 2GB limit. I'll have a think about it.
I was trying to Serialize a large R*Tree using the APIs in this Library, but it failed with error :
Exception in thread "main" java.lang.AssertionError: FlatBuffers: cannot grow buffer beyond 2 gigabytes. at com.google.flatbuffers.FlatBufferBuilder.growByteBuffer(FlatBufferBuilder.java:127) at com.google.flatbuffers.FlatBufferBuilder.prep(FlatBufferBuilder.java:173) at com.google.flatbuffers.FlatBufferBuilder.addOffset(FlatBufferBuilder.java:291) at com.google.flatbuffers.FlatBufferBuilder.addOffset(FlatBufferBuilder.java:562) at com.github.davidmoten.rtree.fbs.generated.Entry.addObject(Entry.java:33) at com.github.davidmoten.rtree.fbs.generated.Entry.createEntry(Entry_.java:26) at com.github.davidmoten.rtree.fbs.FlatBuffersHelper.addEntries(FlatBuffersHelper.java:77) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:96) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102) at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.write(SerializerFlatBuffers.java:73)
Seems like FlatBuffers can't serialize large structures with size more than 2GB? Am I mistaken here? Does this implementation support any other serialization apart from FlatBuffer? I saw few test cases for Kryo Serialization but don't think if the current structure supports this. Please guide.