search

HKUDS / LightRAG

"LightRAG: Simple and Fast Retrieval-Augmented Generation"
https://arxiv.org/abs/2410.05779
MIT License
9.22k stars 1.13k forks source link

Testing on about 1000 documents and more #312

Open kabani7 opened 1 day ago

kabani7 commented 1 day ago

Hello, I want to test this solution on about 1000 document and more, has anybody test it on that amount of data?. if not does anyone has any recommendations on how to do it like do I need to change the vector store and KG database?, is neo4j or Oracle 23ai better on this big scale?, any other recommendation? .

GaryDean commented 1 day ago

i have done it on 3500+ txt/md files, of mixed sizes. very slow, but no problems.

kabani7 commented 1 day ago

i have done it on 3500+ txt/md files, of mixed sizes. very slow, but no problems.

did you use Neo4j or the default KG?, also did you change the Vector database ?

GaryDean commented 20 hours ago

just the default. minor additions to vector database.