timothycarambat
commented
1 month ago https://docs.anythingllm.com/llm-not-using-my-docs#vector-database-settings--max-context-snippets
That is how RAG works on a fundamental level. If you want all 20 PDFs to be sent (which we highly do not recommend and wont even work unless using Claude or something with a massive context window) you can pin the documents.
RAG splits the documents up and per-query retrieves what are semantically determined to be the best chunks to answer the query.
How are you running AnythingLLM?
AnythingLLM desktop app
What happened?
Hi, im running local with LM Studio and in house RAG. I just added 20 small PDFs to the workspace and im noticing that only 3-5 of them are considered in queries. The number of vectors is realistic for 20 documents but it seems when i ask about specific documents only very few of them are considered.
kind regards F
Are there known steps to reproduce?
No response