The re-ranked chunks are incorrect when using the nvidia/rerank-qa-mistral-4b model for reranking. Typically, the second least relevant chunk is mistakenly placed first, while the second chunk (which should be the most relevant) is ranked lower than it should be.
Expected behavior
The chunks should be returned in the same order they are received from the Nvidia API endpoint.
Steps to reproduce
1. Upload some documents, and preprocess them.
2. Enter the API key to RAGflow from https://build.nvidia.com/nvidia/rerank-qa-mistral-4b
3. Run retrieval test.
Additional information
rerank_model.py seems to be working correctly, the problem is most likely in rag/nlp/search.py.
Is there an existing issue for the same bug?
Branch name
v0.11.0
Commit ID
2f33ec7ad07db037482ef5cfa58df1b3dd0727a5
Other environment information
Actual behavior
The re-ranked chunks are incorrect when using the
nvidia/rerank-qa-mistral-4bmodel for reranking. Typically, the second least relevant chunk is mistakenly placed first, while the second chunk (which should be the most relevant) is ranked lower than it should be.Expected behavior
The chunks should be returned in the same order they are received from the Nvidia API endpoint.
Steps to reproduce
Additional information
rerank_model.pyseems to be working correctly, the problem is most likely inrag/nlp/search.py.