[ECS] [feature]: Amazon ECS enhanced capacity management

[AbhishekNautiyal]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

Amazon ECS will deliver a new capacity management solution that combines the simplicity of AWS Fargate’s operational model with the breadth of capabilities supported by Amazon EC2 instances. Customers will be able to use the same simple interface as Fargate by only needing to request vCPU and memory for their tasks, with ECS automatically provisioning, patching, scaling, and cost-optimizing fully-managed compute. At the same time, customers who need more control, will have the ability to select specific EC2 instance types and sizes. Customers can also enable privileged Linux capabilities, and run their desired security and monitoring agents as Daemons. Salient features of enhanced capacity management:

• Run workloads requiring specific compute capabilities, such as accelerated compute (GPUs), CPU instruction sets, network-optimized or burstable instances, while retaining the operational benefits of AWS-managed compute infrastructure and automatic security patching
• Leverage breadth of EC2 billing and purchase options for fully-managed compute, including Reserved Instances, Spot Instances, Compute and Instance Savings Plans, On-demand Capacity Reservations, and Capacity Blocks for ML
• Rapid and cost-effective autoscaling with smart task placement and rebalancing to ensure your applications run on highly available, performant, and cost-efficient compute
• Improved security posture with EC2 instances preconfigured with AWS security best practices and automatically updated with the latest security patches. Use event windows for more control over instance patching and updates
• Run managed Daemon sets, and optionally configure privileged Linux capabilities, including CAP_NET_ADMIN, CAP_SYS_ADMIN, CAP_BPF, etc., to run your desired monitoring, observability, and security solutions

Popular issues that will be addressed by this capability

AWS Fargate GPU Support: When is GPU support coming to fargate? [Fargate] [request]: compute optimized options [Fargate] [request]: Burstable CPU [Fargate] [request]: offer high-performance network options #715

[Fargate] [request]: Allow privileged mode

[Rebalancing] Smarter allocation of ECS resources ECS tasks re-balancing on autoscaling

[ECS] Full support for Capacity Providers in CloudFormation

Which service(s) is this request for? Amazon ECS, AWS Fargate

Amazon ECS team is excited to build this feature, and we look forward to your feedback.

[r-heimann]

Awesome to see all these features being worked on! As an avid Fargate user, it would be great if the issue https://github.com/aws/containers-roadmap/issues/938 could also get some love. If you are using EC2 with ECS, you can do something like (CloudFormation)

          LinuxParameters:
            Tmpfs:
              - ContainerPath: /tmp
                MountOptions:
                  - rw
                Size: 200

that is not possible with Fargate. You cannot specify specific mounting options:

          MountPoints:
            - SourceVolume: tmp
              ContainerPath: /tmp

In the above example the /tmp folder will be mounted by root, unless the docker image has this as a volume with another user configured. Enhancing this feature would make Fargate so much more useful.