In the k6 test suite, there is a delay 60 secs step between rest/restjava and restjava/web3
HPAs for rest/restjava/web3 all have a default stabilization window of 300 secs for scaling down, which means, even though the current desired number of replicas would require the replicaset to scale down sharply, it would not happen or scale down to the lower number if there are previous high numbers in the past 300 secs window
Combined with the max number of compute nodes for the node pool, it's possible that the ramp up stage for restjava/web3 will have a hard time to make the corresponding HPA to increase the restjava/web3 pods because there are still too many rest/restjava pods taking up resources in the 300s stabilization window.
Problem
Background:
HPAs for rest/restjava/web3 all have a default stabilization window of 300 secs for scaling down, which means, even though the current desired number of replicas would require the replicaset to scale down sharply, it would not happen or scale down to the lower number if there are previous high numbers in the past 300 secs window
Combined with the max number of compute nodes for the node pool, it's possible that the ramp up stage for restjava/web3 will have a hard time to make the corresponding HPA to increase the restjava/web3 pods because there are still too many rest/restjava pods taking up resources in the 300s stabilization window.
Solution
Alternatives
No response