search

HKUDS / LightRAG

"LightRAG: Simple and Fast Retrieval-Augmented Generation"
https://arxiv.org/abs/2410.05779
MIT License
9.22k stars 1.13k forks source link

What is the maximum txt file size? #294

Open derekcbr opened 4 days ago

derekcbr commented 4 days ago

I have a txt file with size at 20M, it halted during the running process by using ollama qwen2.5.

LarFii commented 3 days ago

The processing speed for large files is highly dependent on the hardware. However, based on our experience, local models tend to be very slow when handling such tasks. If you need to process large files like this, it might be more efficient to use an API instead.