search

claranet / terraform-aws-asg-instance-replacement

Terraform module for AWS ASG instance replacement
MIT License
9 stars 10 forks source link

ASG cannot rebalance instances #2

Open raymondbutcher opened 6 years ago

raymondbutcher commented 6 years ago

There is an issue with rebalancing availability zones. A min=2 max=2 ASG had 2 instances in a single zone (perhaps there was previously an outage, whatever) and then it decided to rebalance the instances by starting a 3rd one in a different zone. I guess it would terminate one of the older instances afterwards.

The Lambda function saw 3 instances in a max=2 ASG and immediately terminated the new instance. This happened repeatedly because the ASG kept trying to rebalance them, and the Lambda function kept stopping it.

andrewjeffree commented 6 years ago

There's no nice way of checking if a rebalance is in progress.

The only way I've been able to find is to describe-scaling-activities and then parse the cause of each entry for the string 'An instance was launched to aid in balancing the group's zones' which is kinda messy...