[Master]Pre filter support through custom scoring

[vamshin]

Issue #, if available: https://github.com/opendistro-for-elasticsearch/k-NN/issues/50 Parent PR:- https://github.com/opendistro-for-elasticsearch/k-NN/pull/196/

• As of today k-NN search queries in combination with filters/boolean queries always work as a post filter search. k-Nearest Neighbors are evaluated first and then other queries are run on top of the returned k results . When k is small, it is possible we end up less than required k Neighbors after applying the filter.
• This issue becomes more evident especially when k-NN search is combined with filter queries targeting smaller subset of vectors in the large data set.

Description of changes: We define our own custom Script Engine which uses the language name “knn”. It implements a script class called VectorScoreScript which would be called to override each document’s score with the chosen similarity space like Euclidean, Cosine, Dot product etc. Customer could invoke the script using the “script_score” functionality by passing the language name as “knn” and required parameters.

• The script_score functionality can be incorporated by extending ‘ScriptPlugin’.
• script_score is evaluated for the filtered documents at segment level inside a shard.
• Customers would use size attribute to define the ‘k’.
• script would be compiled once on the first request and cached for later

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[codecov[bot]]

Codecov Report

Merging #224 into master will decrease coverage by 0.95%. The diff coverage is 66.95%.

@@             Coverage Diff              @@
##             master     #224      +/-   ##
============================================
- Coverage     77.61%   76.66%   -0.96%     
- Complexity      308      327      +19     
============================================
  Files            50       53       +3     
  Lines          1193     1307     +114     
  Branches        101      120      +19     
============================================
+ Hits            926     1002      +76     
- Misses          219      243      +24     
- Partials         48       62      +14     

Impacted Files	Coverage Δ	Complexity Δ
...oforelasticsearch/knn/index/util/KNNConstants.java	0.00% <ø> (ø)	0.00 <0.00> (ø)
...arch/knn/plugin/script/KNNScoringScriptEngine.java	44.44% <44.44%> (ø)	3.00 <3.00> (?)
...search/knn/plugin/script/KNNVectorScoreScript.java	61.90% <61.90%> (ø)	4.00 <4.00> (?)
...lasticsearch/knn/plugin/script/KNNScoringUtil.java	78.04% <78.04%> (ø)	11.00 <11.00> (?)
...endistroforelasticsearch/knn/plugin/KNNPlugin.java	100.00% <100.00%> (ø)	11.00 <2.00> (+1.00)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 698c2d8...cbfa05e. Read the comment docs.