tazarov
commented
1 month ago hey @tkg61, the chart in current state runs single-node Chroma. The downside of this is lack of distributed storage support.
Single-node Chroma relies on HNSW and sqlite3 for its embeddings and metadata storage, none of those are process-safe, e.g. multiple replicas reading/writing to the same set of files.
Hosted and distributed Chroma has recently been released. What I can do is to include a distributed Chroma in this chart which will solve your problem, although distributed Chroma does not have HA in the traditional sense of the word.
Another approach in a scenario where active-passive HA is an ok approach then I can help you with a Envoy plugin that can manage the replication of the traffic to one or more replicas.
There's also a storage-based replication that can be setup with something like longhorn so that your storage has a secondary replica on another node where in the event of failure of primary Chroma server you can then fallback to by attaching the replicated storage.
Describe the problem
i was assuming if i used this chart it would setup the environment to where if more than 1 replica was in use, they would be synced to each other. it seems that they are not, i would like to have this documented so that others do not go down a rabbit hole like i did.
Describe the proposed solution
Document that multi-replicas is not the same as a high availability cluster and it is up to the developer to sync the databases manually
Alternatives considered
No response
Importance
I cannot run the chart without it
Additional Information
No response