Handles truncated boundary keys

ruweih commented 3 years ago

Boundary keys might not be real keys:

https://github.com/apple/foundationdb/issues/3608

We need handle those truncated keys, or it will fail in FoundationDB 6.x client:

java.lang.IllegalArgumentException: No terminator found for bytes starting at 1
    at com.apple.foundationdb.tuple.TupleUtil$DecodeState.findNullTerminator(TupleUtil.java:98)
    at com.apple.foundationdb.tuple.TupleUtil.decode(TupleUtil.java:438)
    at com.apple.foundationdb.tuple.TupleUtil.unpack(TupleUtil.java:676)
    at com.apple.foundationdb.tuple.Tuple.fromBytes(Tuple.java:526)
    at com.apple.foundationdb.subspace.Subspace.unpack(Subspace.java:231)
    at org.janusgraph.diskstorage.foundationdb.FoundationDBKeyValueStore.lambda$getBoundaryKeys$1(FoundationDBKeyValueStore.java:264)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at org.janusgraph.diskstorage.foundationdb.FoundationDBKeyValueStore.getBoundaryKeys(FoundationDBKeyValueStore.java:264)

ruweih commented 3 years ago

This is very difficult to be reproduced as mentioned in the original post: https://github.com/apple/foundationdb/issues/3608

We did not see the issue until when the data volume grows to really huge, and it still not guarantee to be reproducible. This is not used by JG storage adapter in Gremlin queries currently, only used by bulk operations.

ruweih commented 3 years ago

Could someone please merge this? It's almost 5 month since approved.

rngcntr commented 3 years ago

I completely forgot about this PR. Merging it now

JanusGraph / janusgraph-foundationdb

Handles truncated boundary keys #56