[ECS] [Feature Suggestion]: Capacity Providers: Managed ASGs

[pavneeta]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Sharing a new Capacity Providers Feature Idea for Feedback from the Github Community.

Tell us about your request ECS should manage the lifecycle of the ASGs associated with the Capacity Providers. Including the AMI updates and Autoscaling.

Which service(s) is this request for? ECS with EC2 Launch type

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? Users will get the Managed compute experience while retaining the observability of the underlying ASG instances. This will reduce the effort required to set up ECS clusters with the required compute Infrastructure and coupling the service and cluster Autoscaling. User can use Managed ASGs through ECS APIs and ECS will provision, manage Autoscaling policies, AMI updates and the lifecycle of the underlying ASGs powering their capacity providers. Since ECS provisions and controls the underlying Instances, ECS will be able to provide better draining and scaling workflows. Users also get he observability required for compliance reasons as the ASGs - Instances are in the customer accounts.

What will the customer Experience look like?

1. As part of thecreate-capacity-providers API customers can select the option of --managed-ASG
2. they can define additional configurations like instance type and Launch config - either within the capacity providers API or point to launch config (TBD)
3. ECS takes over the creation and managed of the AGS
4. Customers can see the ASG in their own accounts but it is largely read only , unless through ECS APIs
5. ECS provides the Automated AMI upgrade Management feature as part of this which allows customers to update their AMI using a 'Rolling Update' style refresh of the underlying EC2 Instances.

Please provide feedback with as much detail about your cases and feature request as a possible.

[sanchitaittech]

Managed scaling option not available in the Sao Paulo(sa-east-1) region

[anoopkapoor]

Managed scaling option not available in the Sao Paulo(sa-east-1) region

Cluster auto scaling is now available in the Sao Paulo region as of June 12. https://aws.amazon.com/about-aws/whats-new/2020/06/amazon-ecs-cluster-auto-scaling-available-additional-regions/

[nitrocode]

Would this feature be covered with the instance refresh feature as long as amazon integrated ECS draining at some point in their roadmap ?

https://aws.amazon.com/blogs/compute/introducing-instance-refresh-for-ec2-auto-scaling/

[Vlaaaaaaad]

FYI: AWS just launched bottlerocket-ecs-updater which safely and gracefully updates nodes using Bottlerocket as their OS!

I think that'll solve the AMI upgrades for most of my usecases 🤞

[ankon]

I guess this has been implemented? https://docs.aws.amazon.com/AmazonECS/latest/developerguide/asg-capacity-providers.html

[Nevon]

ASG capacity providers only solve the scaling of ASGs to accommodate tasks in a cluster. It doesn't solve the lifecycle management of the ASG and facilitate things like instance draining or the equivalent of EC2 ASG instance refresh. To do things like roll out a new AMI to your ASG(s) you still have to manage the instance replacement yourself.

[degould-mm]

Just happened on this problem as well. If the capacity provider uses managed scaling the ASG needs scale-in protection turned on, so we can't use instance refresh to rolling update to the latest AMI. Best thing I can think of doing is scaling out the ECS service, then scale back in to get the older instances to terminate.