search

infiniflow / ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
https://ragflow.io
Apache License 2.0
24.61k stars 2.39k forks source link

[Question]: When to use reranker model? #3754

Open southkorea2013 opened 5 days ago

southkorea2013 commented 5 days ago

Describe your problem

Hi Experts,

I set up Xinference to serve a reranker model bge-reranker-v2-m3, and I can add it to RagFlow successfully.

image image

But during the RAG conversation, I cannot see the reranker calling trace info from the Xinference console.

Any feedback will be appreciated.

Nan

2024-11-29 17:28:44,700 transformers.modeling_utils 1307 INFO     loading weights file /Users/*/.xinference/cache/bge-reranker-v2-m3/model.safetensors
2024-11-29 17:28:44,851 transformers.modeling_utils 1307 INFO     All model checkpoint weights were used when initializing XLMRobertaForSequenceClassification.
2024-11-29 17:28:44,851 transformers.modeling_utils 1307 INFO     All the weights of XLMRobertaForSequenceClassification were initialized from the model checkpoint at /Users/*/.xinference/cache/bge-reranker-v2-m3.
If your task is similar to the task the model of the checkpoint was trained on, you can already use XLMRobertaForSequenceClassification for predictions without further training.
2024-11-29 17:28:44,854 transformers.tokenization_utils_base 1307 INFO     loading file sentencepiece.bpe.model
2024-11-29 17:28:44,854 transformers.tokenization_utils_base 1307 INFO     loading file tokenizer.json
2024-11-29 17:28:44,854 transformers.tokenization_utils_base 1307 INFO     loading file added_tokens.json
2024-11-29 17:28:44,854 transformers.tokenization_utils_base 1307 INFO     loading file special_tokens_map.json
2024-11-29 17:28:44,854 transformers.tokenization_utils_base 1307 INFO     loading file tokenizer_config.json
2024-11-29 17:28:45,049 xinference.core.worker 137 INFO     [request 532c9d6e-ae34-11ef-843b-02ba33e0cec4] Leave launch_builtin_model, elapsed time: 2 s
KevinHuSh commented 2 days ago

It will be invoked during retrieval. You can test it in 'retrival test' of KB configuration pages.

southkorea2013 commented 2 days ago

Thanks Kevin. I cannot see the calling trace from Xinference, so I suspect if the reranker is called during RAG phrase.