AnswerDotAI / rerankers

A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.
Apache License 2.0
1.04k stars 57 forks source link

❗️ `FlashRankRanker` ranking is incorrect – it returns the documents in input order #39

Open lsorber opened 1 month ago

lsorber commented 1 month ago

The ranking produced by FlashRankRanker is incorrect. It returns the documents in input order.

This statement in flashrank_ranker.py^1

        ranked_results = [
            Result(
                document=docs[idx],  # Returns documents in input order.
                score=result["score"],
                rank=idx + 1,
            )
            for idx, result in enumerate(flashrank_results)
        ]

should be:

        ranked_results = [
            Result(
                document=docs[result["id"]],  # Returns reranked documents.
                score=result["score"],
                rank=idx + 1,
            )
            for idx, result in enumerate(flashrank_results)
        ]