We're currently implementing RAG (actually, tested and it works!) in current generation pipeline.
Although it works, the document fragments would bulk the prompt and make it very big, hence it cause serious performance problems.
Furthermore, since the project is aimed for running solely on Mac (as the llama.cpp project does), we're still finding ways to load the embedding model.
NatLee
commented
4 months ago
We're currently implementing RAG (actually, tested and it works!) in current generation pipeline. Although it works, the document fragments would bulk the prompt and make it very big, hence it cause serious performance problems.
Furthermore, since the project is aimed for running solely on Mac (as the llama.cpp project does), we're still finding ways to load the embedding model.
Hence, we hide RAG interfaces from web-ui in current branch, but we still keeps the document parser logic for future use. The https://github.com/h-alice/llamacpp-webui/blob/llamacpp-port/document_rag_processor.py is the RAG document split and retrieve logic. And those files in https://github.com/h-alice/llamacpp-webui/tree/llamacpp-port/doc_parser are document parsers, just to note if in case.