search

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes
Apache License 2.0
18 stars 10 forks source link

Support HA with LeaderElection #23

Closed kerthcet closed 1 week ago

kerthcet commented 2 months ago

We only have one replica for controller right now.

kerthcet commented 2 months ago

/kind feature /help

qinguoyi commented 1 week ago

https://github.com/InftyAI/llmaz/blob/b6141a4d32172f2f6500afdd483958c5407797c6/cmd/main.go#L77-L94

hi, from the code point of view, LeaderElection is currently supported to support HA. Are there any other requirements? would you mean to add more replica nums?

kerthcet commented 1 week ago

Thanks @qinguoyi for pointing it out, that's true. Then what we left is just make it configurable with helm chart, as well as the Replicas of the controller, here's the values.yaml: https://github.com/InftyAI/llmaz/blob/main/chart/values.global.yaml

Would you like to finish the work? That would be really appreciate, I think it wouldn't take a lot of time.

qinguoyi commented 1 week ago

Thanks @qinguoyi for pointing it out, that's true. Then what we left is just make it configurable with helm chart, as well as the Replicas of the controller, here's the values.yaml: https://github.com/InftyAI/llmaz/blob/main/chart/values.global.yaml

Would you like to finish the work? That would be really appreciate, I think it wouldn't take a lot of time.

tks, i push a PR here,https://github.com/InftyAI/llmaz/pull/168

kerthcet commented 1 week ago

Solved by #168

/close