opensearch-project / k-NN

🆕 Find the k-nearest neighbors (k-NN) for your vector data
https://opensearch.org/docs/latest/search-plugins/knn/index/
Apache License 2.0
152 stars 113 forks source link

KNN80DocValues should only be considered for BinaryDocValues fields #2147

Closed VijayanB closed 1 day ago

VijayanB commented 2 days ago

Description

KNN80DocValuesProducer should consider adding files from fields that has BinaryDocValues and doesn't have vector values in producer. By default, k-NN field types post 2.17 don't create BinaryDocValues for knn field anymore. However, users can set doc_values = true, to create BinaryDocValues explicitly like any other field, though it is not used for approximate search. Hence, we want to avoid considering fields that is created using NativeEngines990KnnVectorsFormat. Checking doc Type not equal to BinayDocValues and field doesn't have vectors will make sure we don't accidentally add native engine files that are created from NativeEngines990KnnVectorsFormat.

Related Issues

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

navneet1v commented 2 days ago

@VijayanB thanks for catching and fixing this bug.

navneet1v commented 2 days ago

@VijayanB I won't recommend doing skip-change log for this and add the entry in the change log.

VijayanB commented 2 days ago

@VijayanB I won't recommend doing skip-change log for this and add the entry in the change log.

Ack