Migrating vector serialization functions from sun.misc.Unsafe to java.nio.ByteBuffer.
The new serialization and deserialization are both compatible with sun.misc.Unsafe, so users do not need to worry about re-indexing when they make an upgrade.
The motivations for this switch are:
sun.misc.Unsafe is deprecated and will be removed from the JVM at some point.
sun.misc.Unsafe requires extra security policies to install Elastiknn as a plugin. With ByteBuffer, we're able to remove these policies.
The ByteBuffer API is slightly nicer.
Micro benchmarks show the new functions are roughly as fast or faster across the board:
Related Issue
263
Changes
Migrating vector serialization functions from
sun.misc.Unsafe
tojava.nio.ByteBuffer
.The new serialization and deserialization are both compatible with sun.misc.Unsafe, so users do not need to worry about re-indexing when they make an upgrade.
The motivations for this switch are:
Micro benchmarks show the new functions are roughly as fast or faster across the board:
The only one that's slower is
readInt
, but that is only used in tests anyways.The ann-benchmarks also look good:
Testing and Validation
Standard CI and JMH benchmarking. New unit tests also verify forwards/backwards compatibility with the old serialization format.