grafana / rollout-operator

Kubernetes Rollout Operator
Apache License 2.0
132 stars 17 forks source link

prepare-downscale webhook should check for rollouts #82

Closed jhalterman closed 11 months ago

jhalterman commented 1 year ago

This PR updates the prepare-downscale webhook to check for rollouts that are in progress in the rollout-group, by comparing the statefulset's Status.UpdatedReplicas to the number of ready and running pods in the group.

This PR also:

Fixes #79

jhalterman commented 11 months ago

Tested in dev-01 by attempting to manually downscale a statefulset during a rollout:

kubectl edit sts/ingester-zone-a error: statefulsets.apps "ingester-zone-a" could not be patched: admission webhook "prepare-downscale-cortex-dev-01.grafana.com" denied the request: downscale of statefulsets/ingester-zone-a in cortex-dev-01 from 6 to 4 replicas is not allowed because statefulset ingester-zone-b has 0 non-updated replicas and 6 non-ready replicas You can run kubectl replace -f /var/folders/c6/z230vgsx5rb3hd3vtj2bc1ww0000gn/T/kubectl-edit-1712830600.yaml to try this update again.

Then a few minutes later, once the pods were ready:

kubectl edit sts/ingester-zone-a statefulset.apps/ingester-zone-a edited