search

opensearch-project / k-NN

🆕 Find the k-nearest neighbors (k-NN) for your vector data
https://opensearch.org/docs/latest/search-plugins/knn/index/
Apache License 2.0
152 stars 113 forks source link

Integrate KNNVectorValues with vector ANN Search flow #1952

Closed navneet1v closed 1 month ago

navneet1v commented 1 month ago

Description

Integrate KNNVectorValues with vector ANN Search flow. I have not integrated KNNVectorValues in ScriptScore as currently I am doing more deep-dive to see if we really need it. Will followup on that in next PR.

What changes are present:

  1. While doing exact search during Efficient filtering I have switched from using BinaryDocValues to KNNVectorValues.
  2. Added some utility functions which are also part of this PR: https://github.com/opensearch-project/k-NN/pull/1950. Whichever PR gets merged first the conflicts can be resolved based on that PR.

Related Issues

Resolves #1853

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

navneet1v commented 1 month ago

@jmazanec15 can you review code again. I added a new UT for Factory which was not present.

opensearch-trigger-bot[bot] commented 1 month ago

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-1952-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a4697f4e4e940d7faedb4fd582cf411115fc792c
# Push it to GitHub
git push --set-upstream origin backport/backport-1952-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1952-to-2.x.

navneet1v commented 1 month ago

Doing a manual backport.