This PR introduces a backwards-incompatible change to Voyager: we now prepend a custom metadata header to the start of every index file, indicating three properties that otherwise need to be specified at load time:
number of dimensions
space type (Euclidean, Cosine, Inner Product)
storage data type (Float32, Float8, E4M3)
This allows us to finally just do:
index = voyager.Index.load("my_index.voy")
...instead of:
index = voyager.Index.load(
"my_index.hnsw",
num_dimensions=128,
space=voyager.Space.Euclidean,
storage_data_type=voyager.StorageDataType.Float32
)
(and similar in Java).
This PR also includes a collection of V0 test indices (to ensure backwards compatibility) in the Python tests.
This PR introduces a backwards-incompatible change to Voyager: we now prepend a custom metadata header to the start of every index file, indicating three properties that otherwise need to be specified at load time:
This allows us to finally just do:
...instead of:
(and similar in Java).
This PR also includes a collection of V0 test indices (to ensure backwards compatibility) in the Python tests.
Paired with @dylanrb123.