search

VictoriaMetrics / operator

Kubernetes operator for Victoria Metrics
Apache License 2.0
436 stars 146 forks source link

Add support for horizontalPodAutoscaler in related Helm charts #1164

Open fmunteanu opened 3 days ago

fmunteanu commented 3 days ago

While using victoria-metrics-k8s-stack chart, I'm forced to constantly fine-tune the 3 vmcluster component related resources. I see hpa.go is already present, I think is a matter of simply enabling it into victoria-metrics-k8s-stack Helm chart?

For example, vmstorage memory usage oscillates between 800Mi-3Gi, forcing me to define a very large resource limit to avoid OOM's. Enabling autoscaling should address the issue, I can set the vmstorage resource memory limit to 1Gi and the autoscaling will take care of the rest.

Please see a suggested implementation example.

f41gh7 commented 3 days ago

Hello, it makes no sense to use horizontal auto-scaling with vmstorage. It will only lead to cluster instability and metrics loss. It's the main reason why operator supports HPA only for - vminsert, vmselect and vmagent ( it also should be added to vmauth later).

I think VerticalPodAutoscaler could be really useful for vmstorage, when in-place resource resizing will be stable https://kubernetes.io/blog/2023/05/12/in-place-pod-resize-alpha/

fmunteanu commented 2 days ago

@f41gh7 thank you for the explanation. Is is possible to extend the HPA usage for vminsert, vmselect and vmagent within victoria-metrics-k8s-stack chart?