search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.49k stars 987 forks source link

Examine adding more off-heap vector scoring #13515

Open ChrisHegarty opened 1 month ago

ChrisHegarty commented 1 month ago

This issue has been filed to give consideration to the possibility of adding more off-heap vector scorers.

We recently added a MemorySegment Vector scorer for scoring byte vectors without copying on-heap - scorer loads values directly from the backing memory segment when available. Benchmarks show an approx 2x performance improvement by avoiding copying the vector data on-heap.

Floats

Loading vector data from float[] (the fallback), is only supported in JDK 22 - which might be fine. See https://bugs.openjdk.org/browse/JDK-8318678

Scalar quantized

Each of the possible quantized vector element sizes can be evaluated separately.

relates #13339

benwtrent commented 1 month ago

This relates to: https://github.com/apache/lucene/pull/13497