weighted reciprocal rank fusion

[andersfylling]

Description

The current implementation of retrievers follows closely to the original RRF (Reciprocal Rank Fusion) paper. I wonder if weighting could be introduced such that one result set has a higher/lower weight than the others. eg. 0.75 lexical and 0.25 semantic. This feature can also be found in other products (but not limited to):

• azure ai search: https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking#weighted-scores
• weaviate: https://weaviate.io/developers/weaviate/search/hybrid#balance-keyword-and-vector-search

Suggested API

Current Elasticsearch request:

GET example-index/_search
{
    "retriever": {
        "rrf": { 
            "retrievers": [
                {
                    "standard": { 
                        "query": {
                            "term": {
                                "text": "shoes"
                            }
                        }
                    }
                },
                {
                    "knn": { 
                        "field": "vector",
                        "query_vector": [1.25, 2, 3.5],
                        "k": 50,
                        "num_candidates": 100
                    }
                }
            ],
            "window_size": 50,
            "rank_constant": 20
        }
    }
}

With weights (could be named anything else):

GET example-index/_search
{
    "retriever": {
        "rrf": { 
            "retrievers": [
                {
                    "standard": { 
                        "query": {
                            "term": {
                                "text": "shoes"
                            }
                        }
                    },
                    "weight": 0.25
                },
                {
                    "knn": { 
                        "field": "vector",
                        "query_vector": [1.25, 2, 3.5],
                        "k": 50,
                        "num_candidates": 100
                    },
                    "weight": 0.75
                }
            ],
            "window_size": 50,
            "rank_constant": 20
        }
    }
}

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)