Improved EC2 Spot Instances best practices support in Kubernetes Cluster Autoscaler

[schmutze]

We are researching possibilities of improving support for EC2 Spot Instances best practices within Kubernetes Cluster Autoscaler.

We'd love to hear your requirements and feedback as we research this project!

[edlee2121]

Would like support for launch templates in cluster-autoscaler. In general make it easier to specify multiple sizes/families of instances including spot and on-demand.

This is very important to ensure that spot instances that are terminated due to unavailability are quickly replaced with spot instances of a different size/family or with on-demand instances.

[bruecktech]

cluster-autoscaler currently does not support fallback to on-demand very well. We have two node groups, one for on-demand and another one for spot. When spot nodes are interrupted and a high spot price prevents the launch of new instances in an ASG it takes up to 15 minutes (default setting) until it marks the spot node group as unhealthy and uses the on-demand node group. The reason is that it handles this on a higher level where it just waits if a new node shows up. It would be better if cluster-autoscaler would be aware that the spot node group currently cannot provide any capacity because of the spot price.

[cep21]

Hi,

I think this ticket is very related to this issue: https://github.com/kubernetes/autoscaler/issues/3241