aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.41k stars 3.8k forks source link

aws-iam: Instance Profile default role breaks ECS Service Connect agent #28594

Open ianzylstra opened 6 months ago

ianzylstra commented 6 months ago

Describe the bug

In ECS services using an EC2 capacity provider strategy, an instance profile supplied to the launch template used in the auto scaling group places a condition on the ecs:Poll action in the instance profile's role that prohibits the Service Connect sidecar container from properly initializing.

Expected Behavior

The instance profile should grant eco:Poll to AWS:${Cluster/InstanceProfile/InstanceRole} to container instances.

Current Behavior

The instance profile grants the correct action and resource string, but includes the following condition, which restricts the connection.

"ArnEquals": {                   
  "ecs:cluster": "${Cluster.Arn}"
}

Reproduction Steps

Deploy a Service Connect-configured service with an EC2 capacity provider to an ECS cluster. Tasks will not start due to an unhealthy service connect agent which does not have authorization to poll the ECS management interface.

Possible Solution

Do not restrict the ecs:Poll action.

Additional Information/Context

No response

CDK CLI Version

2.118.0 (build a40f2ec)

Framework Version

aws-cdk-lib@2.118.0

Node.js Version

v20.9.0

OS

macOS 14.2.1

Language

TypeScript

Language Version

TypeScript (5.3.3)

Other information

No response

pahud commented 6 months ago

According the document:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect.html

Container instances must have the ecs:Poll permission for the resource arn:aws:ecs:region:0123456789012:task-set/cluster/*. If you are using the ecsInstanceRole, you don't need to add additional permissions. The AmazonEC2ContainerServiceforEC2Role managed policy has the necessary permissions. For more information, see Amazon ECS container instance IAM role.

Did you see AmazonEC2ContainerServiceforEC2Role managed policy attached on the ecsInstanceRole? If not, I guess we probably should attach this managed policy to the instance role?

ianzylstra commented 6 months ago

Did you see AmazonEC2ContainerServiceforEC2Role managed policy attached on the ecsInstanceRole? If not, I guess we probably should attach this managed policy to the instance role?

@pahud the managed policy was indeed not attached.

It appears as though the default policy gets created via Cluster.configureAutoScalingGroup(). See previous issue a number of years back as impetus for the original down-scoping, but looks as though it might need to be revised?

pahud commented 6 months ago

Yeah I think we need to make sure either the AmazonEC2ContainerServiceforEC2Role is attached or ecs:Poll is added. Before we have a PR for that, I guess it's still possible to attach this managed policy on ecsInstanceRole by ourselves?