Recently we noticed some impacts in the ASG and application Blue/Green deployment via CodeDeploy and an unusual behaviour was displayed, no containers were spin-up by CodeDeploy and no new hosts were starting via ASG.
After applying a manual change in the ASG minimum number of hosts I received the following error from Terraform:
Error: error waiting for Auto Scaling Group Capacity: "ecs-dev-apps": Waiting up to 10m0s: Need exactly 3 healthy instances in ASG, have 1. Most recent activity: {
ActivityId: "6115e1ef-7549-c313-2205-7d86321872b1",
AutoScalingGroupName: "ecs-dev-apps",
Cause: "At 2021-03-29T02:43:22Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 1 to 3.",
Description: "Launching a new EC2 instance. Status Reason: Could not launch Spot Instances. InvalidBlockDeviceMapping - Volume of size 22GB is smaller than snapshot 'snap-0b5c5ea11d58e1fec', expect size >= 30GB. Launching EC2 instance failed.",
Details: "{\"Subnet ID\":\"subnet-0ac816ed857bb9d05\",\"Availability Zone\":\"ap-northeast-1a\"}",
EndTime: 2021-03-29 02:43:25 +0000 UTC,
Progress: 100,
StartTime: 2021-03-29 02:43:25.424 +0000 UTC,
StatusCode: "Failed",
StatusMessage: "Could not launch Spot Instances. InvalidBlockDeviceMapping - Volume of size 22GB is smaller than snapshot 'snap-0b5c5ea11d58e1fec', expect size >= 30GB. Launching EC2 instance failed."
}
After a quick chat with @adenot, we believe the AMI for ECS host increased in size and consequently we need to increase the EBS volume size to handle the new instances.
I did some tests locally and it seems this simple code change solved the issue. After the new launch configuration I was able to retake my usual deployments via CodeDeploy.
Recently we noticed some impacts in the ASG and application Blue/Green deployment via CodeDeploy and an unusual behaviour was displayed, no containers were spin-up by CodeDeploy and no new hosts were starting via ASG.
After applying a manual change in the ASG minimum number of hosts I received the following error from Terraform:
After a quick chat with @adenot, we believe the AMI for ECS host increased in size and consequently we need to increase the EBS volume size to handle the new instances.
I did some tests locally and it seems this simple code change solved the issue. After the new launch configuration I was able to retake my usual deployments via CodeDeploy.
Let me know if you have any additional queries.