search

DeployQL / LintDB

Vector Database with support for late interaction and token level embeddings.
https://www.lintdb.com/
Apache License 2.0
53 stars 2 forks source link

Support Float16 #45

Closed mtbarta closed 3 months ago

mtbarta commented 3 months ago

Since late interaction is already so expensive storage-wise, we can support float16 as a quick win to reduce storage costs.

This would involve adding support for a new datatype. Alternatively, we could consider a quantizer that converts float32 to float16. I'm preferential to a new data type.

mtbarta commented 3 months ago

Let's break this idea up a bit.

We should support float16 for individual FieldValues. For Tensors, quantization should be preferred in reducing storage size.

Also note, we should support float16 computation as well. We can break that out into a separate ticket.