amazon-archives / k8s-cloudwatch-adapter

An implementation of Kubernetes Custom Metrics API for Amazon CloudWatch
Apache License 2.0
158 stars 98 forks source link

Why max out desired number when exceeding the targetvalue by only 1? #13

Closed chuong-dao closed 4 years ago

chuong-dao commented 5 years ago

My targetvalue is set at 6. If I bump up to 7 messages in queue, I get this

  "sqs-helloworld-length" (target value):  7 / 6
Min replicas:                              1
Max replicas:                              10
Deployment pods:                           10 current / 10 desired

That's not good if I set my max to 100. I might at all 100 pods for one single message increase. If I do one lower than the targetvalue, I get this.

  "sqs-helloworld-length" (target value):  5 / 6
Min replicas:                              1
Max replicas:                              10
Deployment pods:                           5 current / 5 desired

Could someone explain why it's behaving this way?

chankh commented 5 years ago

The behavior is controlled by HPA, but this doesn't seem normal to me. HPA is responsible for the scaling, the adapter here is just reports the current value and HPA should decide whether to increase or decrease the number of replicas.

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/

chankh commented 4 years ago

closing this issue since there is no update