[Bug]: Knowledge Based Not Included When Doing Self RAG

infiniflow / ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

Apache License 2.0

23.53k stars 2.31k forks source link

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Branch name

main

Commit ID

5ec19b5

Other environment information

additional context from self rag:
100.82.53.123 - - [19/Jun/2024:17:20:18 -0400] "POST /api/chat HTTP/1.1" 200 935 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22llama3:70b-instruct-fp16\x22, \x22messages\x22: [{\x22role\x22: \x22system\x22, \x22content\x22: \x22\x5Cn        You are an expert at query expansion to generate a paraphrasing of a question.\x5Cn        I can't retrieval relevant information from the knowledge base by using user's question directly.     \x5Cn        You need to expand or paraphrase user's question by multiple ways such as using synonyms words/phrase, \x5Cn        writing the abbreviation in its entirety, adding some extra descriptions or explanations, \x5Cn        changing the way of expression, translating the original question into another language (English/Chinese), etc. \x5Cn        And return 5 versions of question and one is from translation.\x5Cn        Just list the question. No other words are needed.\x5Cn    \x22}, {\x22role\x22: \x22user\x22, \x22content\x22: \x22list cyber security problems that are evidence of a latent security breach\x22}], \x22stream\x22: false, \x22format\x22: \x22\x22, \x22options\x22: {\x22temperature\x22: 0.8}, \x22keep_alive\x22: -1}"
100.82.53.123 - - [19/Jun/2024:17:20:20 -0400] "POST /api/embeddings HTTP/1.1" 200 20625 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22mxbai-embed-large\x22, \x22prompt\x22: \x22Here are five paraphrased questions:\x5Cn\x5Cn1. What are the signs or indicators of an underlying cybersecurity threat?\x5Cn2. List potential warning signals of a previously undetected security compromise\x5Cn3. Identify common manifestations of hidden network vulnerabilities\x5Cn4. What are the typical red flags of an unreported cyber attack?\x5Cn5. \x5Cu00bfCu\x5Cu00e1les son los problemas de seguridad inform\x5Cu00e1tica que sugieren una violaci\x5Cu00f3n de seguridad latente?\x22, \x22options\x22: {}, \x22keep_alive\x22: null}"
100.82.53.123 - - [19/Jun/2024:17:23:17 -0400] "POST /api/chat HTTP/1.1" 200 61892 "-" "ollama-python/0

Actual behavior

I have proxy-ed traffic between ollama and ragflow to detect the issue. it appears when doing self rag, and ollama, questions are generated and submitted, but the knowledge base is empty (I added bold to make it easier to see):

100.82.53.123 - - [19/Jun/2024:17:27:39 -0400] "POST /api/chat HTTP/1.1" 200 10517 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22llama3:70b-instruct-fp16\x22, \x22messages\x22: [{\x22role\x22: \x22system\x22, \x22content\x22: \x22You are an intelligent assistant. Please summarize the content of the knowledge base to answer the question. Please list the data in the knowledge base and answer in detail. When all knowledge base content is irrelevant to the question, your answer must include the sentence \x5C\x22The answer you are looking for is not found in the knowledge base!\x5C\x22 Answers need to consider chat history.\x5Cn Here is the knowledge base:\x5Cn \x5Cn The above is the knowledge base.\x22}, {\x22role\x22: \x22user\x22, \x22content\x22: \x22list cyber security problems that are evidence of a latent security breach\x22}], \x22stream\x22: true, \x22format\x22: \x22\x22, \x22options\x22: {\x22temperature\x22: 0.1, \x22num_predict\x22: 933, \x22top_k\x22: 0.3, \x22presence_penalty\x22: 0.4, \x22frequency_penalty\x22: 0.7}, \x22keep_alive\x22: -1}"

Expected behavior

I would expect there to be knowledge

Steps to reproduce

use llama3
configure with self-rag
ask a question of knowledge base where you know you have relevant content

Additional information

No response

Here is some additional detailed entries from when simply trying to index the data using RAG to summarize content (it appears none of the content is being passed back).

100.80.54.128 - - [20/Jun/2024:10:59:27 -0400] "POST /api/embeddings HTTP/1.1" 200 20585 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22mxbai-embed-large\x22, \x22prompt\x22: \x22There are no paragraphs provided for me to summarize. Please provide the paragraphs you would like me to summarize, and I'll be happy to assist you!\x22, \x22options\x22: {}, \x22keep_alive\x22: null}" 100.80.54.128 - - [20/Jun/2024:10:59:27 -0400] "POST /api/embeddings HTTP/1.1" 200 20585 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22mxbai-embed-large\x22, \x22prompt\x22: \x22There are no paragraphs provided for me to summarize. Please provide the paragraphs you would like me to summarize, and I'll be happy to assist you!\x22, \x22options\x22: {}, \x22keep_alive\x22: null}" 100.80.54.128 - - [20/Jun/2024:10:59:27 -0400] "POST /api/embeddings HTTP/1.1" 200 20684 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22mxbai-embed-large\x22, \x22prompt\x22: \x22There are no paragraphs provided for me to summarize. Please provide the actual text, and I'll be happy to help!\x22, \x22options\x22: {}, \x22keep_alive\x22: null}" 100.80.54.128 - - [20/Jun/2024:11:00:15 -0400] "POST /api/chat HTTP/1.1" 200 745 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22llama3:70b-instruct-fp16\x22, \x22messages\x22: [{\x22role\x22: \x22system\x22, \x22content\x22: \x22You're a helpful assistant.\x22}, {\x22role\x22: \x22user\x22, \x22content\x22: \x22Please summarize the following paragraphs. Be careful with the numbers, do not make things up. Paragraphs as following:\x5Cn There\x5CnThere\x5CnThe above is the content you need to summarize.\x22}], \x22stream\x22: false, \x22format\x22: \x22\x22, \x22options\x22: {\x22temperature\x22: 0.3, \x22num_predict\x22: 1000}, \x22keep_alive\x22: -1}" 100.80.54.128 - - [20/Jun/2024:11:00:24 -0400] "POST /api/chat HTTP/1.1" 200 708 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22llama3:70b-instruct-fp16\x22, \x22messages\x22: [{\x22role\x22: \x22system\x22, \x22content\x22: \x22You're a helpful assistant.\x22}, {\x22role\x22: \x22user\x22, \x22content\x22: \x22Please summarize the following paragraphs. Be careful with the numbers, do not make things up. Paragraphs as following:\x5Cn There\x5CnThe above is the content you need to summarize.\x22}], \x22stream\x22: false, \x22format\x22: \x22\x22, \x22options\x22: {\x22temperature\x22: 0.3, \x22num_predict\x22: 1000}, \x22keep_alive\x22: -1}" 100.80.54.128 - - [20/Jun/2024:11:00:26 -0400] "POST /api/embeddings HTTP/1.1" 200 20627 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22mxbai-embed-large\x22, \x22prompt\x22: \x22I apologize, but there is no content provided for me to summarize. The text only contains three instances of \x5C\x22There\x5C\x22 without any actual paragraphs or information. If you could provide the actual content, I'd be happy to help you with summarizing it!\x22, \x22options\x22: {}, \x22keep_alive\x22: null}" 100.80.54.128 - - [20/Jun/2024:11:00:26 -0400] "POST /api/embeddings HTTP/1.1" 200 20635 "-" "ollama-python/0.1.8 (x86_64 linux) Python/3.10.12" "{\x22model\x22: \x22mxbai-embed-large\x22, \x22prompt\x22: \x22I apologize, but there are no paragraphs provided for me to summarize. The text only says \x5C\x22There\x5C\x22 and then nothing else. If you could provide the actual paragraphs, I'd be happy to help you with summarizing them!\x22, \x22options\x22: {}, \x22keep_alive\x22: null}"

infiniflow / ragflow