search

llmware-ai / llmware

Unified framework for building enterprise RAG pipelines with small, specialized models
https://llmware-ai.github.io/llmware/
Apache License 2.0
6.71k stars 1.52k forks source link

Kubernetes reference implementation #1027

Open doberst opened 1 month ago

doberst commented 1 month ago

LLMWare provides several Docker implementation scripts and a devcontainer reference script.

We would welcome contributions from Kubernetes experts to provide a reference Kubernetes configuration and 'fast start' script to deploy llmware in a Kubernetes cluster as well as advising us on additional steps and capabilities that will facilitate Kubernetes scalable deployments.

This is a great first issue if you are an expert in Kubernetes and just starting to learn llmware.

Lelin07 commented 1 week ago

@doberst I'm planning to work on the Kubernetes deployment for LLMWare. Given the multi-container setup and the complexity of the project, I have a few questions to ensure the solution meets your expectations. Since this is a "good first issue" and for a "reference" Kubernetes configuration, I want to make sure we align on the requirements.

The project would involve deploying multiple services (MongoDB, Milvus, Neo4j, Pgvector, Qdrant, Redis Stack) and managing inter-service communication, resource allocation, configuration management, external access, scalability (HPA) and monitoring/logging.

Follow-Ups:

  1. Scope of Initial Setup: Should the initial Kubernetes configuration focus on a basic setup with Deployments and Services?
  2. Resource Requirements: Are there any specific resource requirements (CPU, memory) that should be considered for the deployments to ensure optimal performance?

Thank you for your time and assistance.