duckdb / duckdb_vss

MIT License
94 stars 10 forks source link

Adding support for binary embeddings #37

Open bhugueney opened 1 week ago

bhugueney commented 1 week ago

Thank you for this most useful extension ! It seems that binary embeddings allow dramatic increase in performance for a small accuracy cost ( https://huggingface.co/blog/embedding-quantization#quantization-experiments ). Various other vector DB allow to use them :

It would be great if DuckDB vss could also support them efficiently.

EDIT pg_vecor also has it : https://github.com/pgvector/pgvector?tab=readme-ov-file#binary-vectors Best Regards

Maxxen commented 1 day ago

Hello!

Yes, support for arrays of other types is planned, although binary vectors in particular might be slightly more complex since DuckDB itself doesn't really have a "bit" type, but it should be doable.