search

Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
https://anythingllm.com
MIT License
26.53k stars 2.65k forks source link

[BUG]: Bad result from RAG #1406

Closed CharlesBdg closed 5 months ago

CharlesBdg commented 5 months ago

How are you running AnythingLLM?

Docker (local)

What happened?

Hello, Since 3 weeks, I get really bad result at RAG. Before around 18th April, I was able to get accurate information from my files (around 150 txt files). Now, the LLM always said he didn't find any relevant context. When I click on "Show citation", the file where the information is, is not showing up. For example : I have a file called "How to enable Warp / Zero Trust" (from Cloudflare). Inside the file it is written "How to enable Warp / Zero Trust". I ask the question to the LLM "How to enable Warp / Zero Trust", Output : "Sorry I didn't find any relevant context" and the file is not in "Show citation".

LLM : Ollama local / llama3, phi3, openchat, mistral, same output Embedding : Ollama / mxbai-embed-large Vector database : LanceDB or Milvus (I've already tried a hard reset of the DB).

I even tested with the Desktop version (v1.5.4) (in case it was a Docker issue) but same issue.

I've already tried tweaking in "Vector Database" section of the Workspace the "Max Context Snippets" and "Document similarity threshold" but no result.

I don't know if something break with the merged of Agent or the bump version in lancedb deps or bump langchain deps or the code that do the RAG.

Maybe add the ability to completely disable the Agent and get back the old RAG.

PS : Great project, thank you

Are there known steps to reproduce?

No response

timothycarambat commented 5 months ago

Agents do not run unless you invoke them, they have no impact on regular RAG chats

CharlesBdg commented 5 months ago

Thanks for the information.

Here what I've found, I have embed the Wikipedia page of OpenAI Default settings on the workspace. Question : "How much Microsoft invested ?" Answer : "Microsoft invested $1 billion in OpenAI, a non-profit artificial intelligence research organization, in 2019. This investment was part of a multi-year partnership aimed at advancing AI capabilities and promoting responsible AI development." Real answer should be : "Microsoft provided OpenAI Global LLC with a $1 billion investment in 2019 and a $10 billion investment in 2023" When I look at the "Citation", I don't even see the answer the LLM gave to me. And every chunk of text have at least "60% match" but in reality, it's totally wrong (see pictures) Capture d’écran du 2024-05-16 00-09-46 Capture d’écran du 2024-05-16 00-08-51 Capture d’écran du 2024-05-16 00-06-07

man2004 commented 5 months ago

I am using gpt-4o and text-embedding-ada-002 in LLM and embedding respectively. Below is my result after adding the Wikipedia page of OpenAI:

Q: How much has Microsoft invested in OpenAI? A: Microsoft has invested a total of $11 billion in OpenAI, with an initial $1 billion investment in 2019 and an additional $10 billion investment in 2023.

CharlesBdg commented 5 months ago

It seems I had two issues :

  1. The embedding model I used has a context size too small (512 tokens), thanks to @man2004 who use text-embedding-ada-002 and has a context of 8192 tokens. I now use nomic-embed-text 8192 tokens with Ollama. It solved 50% of my issue. This website was helpful to understand the embedding size and chunk overlap. https://langchain-text-splitter.streamlit.app/
  2. It seems my AnythingLLM docker and storage/lanceDB was messy, so I have deleted everything and done a clean installation. Which solve the other 50%.

Now everything is back to normal and work has expected. Thank you for your help.