opensearch-project / k-NN

🆕 Find the k-nearest neighbors (k-NN) for your vector data
https://opensearch.org/docs/latest/search-plugins/knn/index/
Apache License 2.0
156 stars 123 forks source link

KNN80DocValues should only be considered for BinaryDocValues fields #2147

Closed VijayanB closed 1 month ago

VijayanB commented 1 month ago

Description

KNN80DocValuesProducer should consider adding files from fields that has BinaryDocValues and doesn't have vector values in producer. By default, k-NN field types post 2.17 don't create BinaryDocValues for knn field anymore. However, users can set doc_values = true, to create BinaryDocValues explicitly like any other field, though it is not used for approximate search. Hence, we want to avoid considering fields that is created using NativeEngines990KnnVectorsFormat. Checking doc Type not equal to BinayDocValues and field doesn't have vectors will make sure we don't accidentally add native engine files that are created from NativeEngines990KnnVectorsFormat.

Related Issues

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

navneet1v commented 1 month ago

@VijayanB thanks for catching and fixing this bug.

navneet1v commented 1 month ago

@VijayanB I won't recommend doing skip-change log for this and add the entry in the change log.

VijayanB commented 1 month ago

@VijayanB I won't recommend doing skip-change log for this and add the entry in the change log.

Ack