opensearch-project / neural-search

Plugin that adds dense neural retrieval into the OpenSearch ecosytem
Apache License 2.0
61 stars 65 forks source link

Block query execution if hybrid query is part of the compound query #517

Open martin-gaievski opened 10 months ago

martin-gaievski commented 10 months ago

Is your feature request related to a problem?

Currently it's possible to include hybrid query as part of the most of compound queries, like function_score. That is a problem as resulting scores are incorrect. This is because hybrid query designed in a way that it must be a top level query.

What solution would you like?

We need to block query execution for cases when Hybrid query is included as part of the compound query . This is already done for Bool query, but not for other queries. This is the list of such queries for OpenSearch https://opensearch.org/docs/latest/query-dsl/compound/index/:

What alternatives have you considered?

Execute queries separately and do normalization outside the opensearch.

Do you have any additional context?

Initial issue came up in scope of https://github.com/opensearch-project/neural-search/issues/466, and has been closed for Bool in a corresponding PR https://github.com/opensearch-project/neural-search/pull/498

nwatab commented 3 months ago

I want this feature. I'd like to query

$$( w{match} \cdot \text{MinMax} (match) + w{kNN} \cdot \text{MinMax} (kNN) ) \cdot \text{decay}(\sigma)$$

, but I can't. I will alternate it with

$$w{match} \cdot \text{MinMax} (match \cdot \text{decay} (\sigma)) + w{kNN} \cdot \text{MinMax} (kNN \cdot \text{decay}(\sigma))$$

, which isn't ideal.