search

HKUDS / LightRAG

"LightRAG: Simple and Fast Retrieval-Augmented Generation"
https://arxiv.org/abs/2410.05779
MIT License
6k stars 609 forks source link

Add Support for Production Database Storage #106

Open proBhavesh opened 4 days ago

proBhavesh commented 4 days ago

Add Support for Production Database Storage

Current State

LightRAG stores vectors and documents locally, which limits production deployment capabilities.

Request

Add support for production databases:

Use Case

We're building a legal research tool and need to:

Questions

  1. Would you be interested in this feature?
  2. Any preferences for database backends?

Happy to contribute to implementation if there's interest.

LarFii commented 3 days ago

We actually really need this feature, but since we're not familiar with databases, we haven't implemented it yet. We truly appreciate your offer to help!

wiltshirek commented 2 days ago

Add Support for Production Database Storage

Current State

LightRAG stores vectors and documents locally, which limits production deployment capabilities.

Request

Add support for production databases:

  • PostgreSQL + pgvector for vector storage
  • Document storage with proper DB backend
  • Connection pooling and async support
  • Basic monitoring

Use Case

We're building a legal research tool and need to:

  • Handle large document collections
  • Support concurrent users
  • Enable cloud deployment
  • Ensure data persistence

Questions

  1. Would you be interested in this feature?
  2. Any preferences for database backends?

Happy to contribute to implementation if there's interest.

I'm looking into using Neo4J to replace NetworkX. Happy to contribute as well. Just need some advice on directed vs undirected graphs. Looks like we should require directed for a production scenario using this algo. Any feedback is helpful.