search

HKUDS / LightRAG

"LightRAG: Simple and Fast Retrieval-Augmented Generation"
https://arxiv.org/abs/2410.05779
MIT License
9.22k stars 1.13k forks source link

error in ollama with local llm #298

Open 2270677671 opened 4 days ago

2270677671 commented 4 days ago

When I download the gguf file of Qwen 2.5 from Hugging Face and deploy it as an LLM for LightRag through Ollama's modelfile, it always gets stuck at the last step, no matter how large or small my txt file is, I always encounter this issue. (rag) root@9748b0cfb85c:~/lanyun-tmp/LightRAG-main/examples# python lightrag_ollama_demo.py INFO:lightrag:Logger initialized for working directory: ./dickens INFO:lightrag:Load KV llm_response_cache with 0 data INFO:lightrag:Load KV full_docs with 0 data INFO:lightrag:Load KV text_chunks with 0 data INFO:nano-vectordb:Init {'embedding_dim': 768, 'metric': 'cosine', 'storage_file': './dickens/vdb_entities.json'} 0 data INFO:nano-vectordb:Init {'embedding_dim': 768, 'metric': 'cosine', 'storage_file': './dickens/vdb_relationships.json'} 0 data INFO:nano-vectordb:Init {'embedding_dim': 768, 'metric': 'cosine', 'storage_file': './dickens/vdb_chunks.json'} 0 data INFO:lightrag:[New Docs] inserting 1 docs INFO:lightrag:[New Chunks] inserting 39 chunks INFO:lightrag:Inserting 39 vectors to chunks INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/embeddings "HTTP/1.1 200 OK" INFO:lightrag:[Entity Extraction]... INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 200 OK" ⠇ Processed 38 chunks, 378 entities(duplicated), 218 relations(duplicated)

LarFii commented 2 days ago

Based on the logs, it seems the insertion process is ongoing, and there doesn’t appear to be an obvious issue. Could you provide more specific details, such as the model size, the size of the text being processed, and how long it remains stuck?

penghd commented 2 days ago

I've encountered the same issue before, and the reason was that ollama load qwen 2.5 with a context size(2048) that is too small to accommodate the extraction prompt. When creating the modelfile, you can add the following line: PARAMETER num_ctx 12800 Same issue with GRAPHRAG.