apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.45k stars 973 forks source link

Protect against nan & inf values in quantizer and test with tiny vectors #13366

Closed benwtrent closed 1 month ago

benwtrent commented 1 month ago

In testing, it was noticed that tiny vectors (<= 2 dims) could cause weird behavior in the quantizer.

Additionally, we should fail any quantization that ends up with NaN or Inf values.