[New Feature][Accuracy] Reranker

[atroyn]

Reranker

Re-ranking is a quick accuracy win, and as cheap computationally as sentence transformers.

Re-ranking can take account of the query as well as additional metadata when evaluating the relevance of results. For example, it’s fairly straightforward to add a weighted term to account for data recency.

It also provides a normalized measure of relevancy, allowing users to more easily filter on the relevancy of results.

API Design

# Get a re-ranker
reranker = chromadb.utils.rerankers.Reranker()

# API 1.

# Pass it as an additional argument to query:
result = collection.query(...., reranker=reranker)

# Return a new field, 'rank_scores' 
result.reranker_scores: List[List[Double]] # Score per retrived document per query

# API 2.
reranker.rerank(results, query_text) # return the query back? 

# API 3. 
results.rerank(reranker) 

[Complexity] Subtask

• [Low] Use a sentence transformer cross-encoder as a basic re-ranker.
• [Low] Implement Cohere re-ranker as a third-party re-ranker.
• [Low] Evaluate the effectiveness of the default re-ranker we ship with the default embedding model to ensure we actually get wins.
• [Med] Ship a metadata-aware re-ranker (use RRF)

[Yimi81]

any update?

[atroyn]

@Yimi81 We expect to release this as part of the current (v.0.6) milestone