DeployQL / LintDB

Vector Database with support for late interaction and token level embeddings.
https://www.lintdb.com/
Apache License 2.0
53 stars 2 forks source link

Add better tests #36

Closed mtbarta closed 5 months ago

mtbarta commented 5 months ago

This PR adds a test on ColBERT.

We compare the scores of the top 10 results within .1 tolerance.

This can be improved. At one point, we had exact parity with ColBERT on CPU and not much has changed. Now, we're comparing LintDB on CPU with ColBERT on GPU.

It's possible there's some divergence in the results now. I need to re-extract a ColBERT branch that can output intermediate steps for comparison.