search

infiniflow / ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
https://ragflow.io
Apache License 2.0
21.85k stars 2.13k forks source link

[Question]: Different chunks returned in RAGFlow vs LangChain #3118

Open rplescia opened 5 days ago

rplescia commented 5 days ago

Describe your problem

I've been testing queries in RAGflow using the embedding model (all-MiniLM-L6-v2) and 'General' method to chunk and embed documents and my ollama (Llama3.1 8B) for LLM inference. I am comparing the results to a streamlit/langchain app that I built myself and I'm getting very different results for the document chucks that are returned as part of the RAG query.

I have been comparing results returned during retrieval testing in RAGflow versus the chunks returned when I do the same search on my solution built on langchain+faiss. I seem to be getting much more accurate results with the langchain+faiss solution. Is there any way I can improve the accuracy of the chunks returned in RAGflow as it is not providing the LLM with the correct chunks at the moment?

KevinHuSh commented 5 days ago

Could you elaberate on your cases? And please try this setup: image

rplescia commented 4 days ago

I am reviewing legal documents like large loan agreements and contracts. I have spent much time trying different chunking methods, delimiters, token numbers, keywords etc. From what I can see the 'problem' is not with the chunking of the documents but rather the results returned by the vector search.

I get similar chucks returned when I ask different questions which means I rarely get the correct answer. I have also noticed that the Vector similarity for the results are 0 so I'm not sure it is even doing a vector search.

image

KevinHuSh commented 4 days ago

Weird! Vector sim can't be 0 anyway. What's your embedding model?

rplescia commented 1 day ago

I thought it was weird too. I'm using the all-MiniLM-L6-v2 model that comes with 'full' build. I can see the document chunks and the embedding seems to run without any error, but when testing retrieval no vector search is performed.

image