search

Cinnamon / kotaemon

An open-source RAG-based tool for chatting with your documents.
https://cinnamon.github.io/kotaemon/
Apache License 2.0
14.64k stars 1.14k forks source link

[BUG] While using complex reasoning method the relevance score is 0.0 always and still it answers the question. #365

Open arya18mak opened 2 weeks ago

arya18mak commented 2 weeks ago

Description

In complex reasoning method, multi-step retrieval process is being used, so which chunks are getting displayed in the information panel, multi-retrieval meaning different sub-questions and chunks retrieval for different cases. So which of them are selected I kind of explored the code but could'nt understand. And relevance score remains zero why so(the answer is still right,content is highlighted saying full text search, which means correct chunks)

Reproduction steps

1. Go to 'complex setting'
2. Click on 'send/regen'
3. Scroll down to 'information panel and relevance score.'
4. See error

Screenshots

![DESCRIPTION](LINK.png)

Logs

No response

Browsers

No response

OS

Windows

Additional information

No response

sachinkumaranumula commented 1 week ago

I faced the same initially then I just looked at the Retrieval Settings and switched to model that I was using which in my case was ollama and not the default openai that will pull relevance score from underlying LLM and get you what you want.

image