opensearch-project / k-NN

🆕 Find the k-nearest neighbors (k-NN) for your vector data
https://opensearch.org/docs/latest/search-plugins/knn/index/
Apache License 2.0
152 stars 113 forks source link

Integration of Quantization Framework for Binary Quantization with Indexing Flow #1996

Closed Vikasht34 closed 1 month ago

Vikasht34 commented 1 month ago

Description

This PR introduces changes to the indexing flow by integrating it with the Quantization Framework, specifically adding support for Binary Quantization. Additionally, the PR includes updates to the Quantization State to provide accurate RAM usage estimates for Quantization State instances.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Vikasht34 commented 1 month ago

This failure in Integ tests for Window is un related to this PR

REPRODUCE WITH: gradlew ':test' --tests "org.opensearch.knn.indices.ModelDaoTests.testCreate" -Dtests.seed=4AD3C3C3F1DEFD7A -Dtests.security.manager=false -Dtests.locale=teo-UG -Dtests.timezone=America/Eirunepe -Druntime.java=21
org.opensearch.knn.indices.ModelDaoTests > testCreate FAILED
    java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([4AD3C3C3F1DEFD7A:759903A4454CD0FB]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.knn.indices.ModelDaoTests.testCreate(ModelDaoTests.java:110)