opensearch-project / neural-search

Plugin that adds dense neural retrieval into the OpenSearch ecosytem
Apache License 2.0
61 stars 64 forks source link

[Backport 2.x] Implement parallel execution of sub-queries for hybrid search (#749) #786

Closed VijayanB closed 3 months ago

VijayanB commented 3 months ago

Description

Add new thread pool to schedule tasks that are related to hybrid query execution Register executor builders with Plugin Use Lucene's Task Executor to execute and collect results Parallelize Query re-write Parallelize score supplier creation Parallelize build hybrid scores

Backported #749 manually

Issues Resolved

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

martin-gaievski commented 3 months ago

Please add [Backport 2.x] in PR header and commit ref in main from where we're cherry picking.

VijayanB commented 3 months ago

Please add [Backport 2.x] in PR header and commit ref in main from where we're cherry picking.

I added in commit. I will add in PR description as well. Thanks for feedback