search

0xDebabrata / citrus

(distributed) vector database
https://searchcitrus.com
Apache License 2.0
104 stars 13 forks source link

Support non-integer IDs #14

Closed 0xDebabrata closed 1 year ago

0xDebabrata commented 1 year ago

citrus only supports integer IDs at the moment because that's passed directly to HNSW. Add support for non-integer IDs by creating a map between each ID and an incrementing integer value which will serve as the HNSW label.

0xDebabrata commented 1 year ago

Instead of creating a map between ID and an integer and saving and loading that from a file, make use of the postgres/sqlite db layer to maintain an auto incrementing integer primary key. Maintaining crucial info through files can get hinky with multithreading.