aws / copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
https://aws.github.io/copilot-cli/
Apache License 2.0
3.42k stars 397 forks source link

Does Copilot Allow Management of Roles #4626

Open zmad5306 opened 1 year ago

zmad5306 commented 1 year ago

We have an app that currently has 53 jobs & services across 6 environments. Copilot creates 4+ roles per environment per job/service. We are running into role quota limits in the AWS account due to the number of roles that Copilot creates. Is there a way to create less roles? How should this be handled?

KollaAdithya commented 1 year ago

Hello @zmad5306 ! Currently we are working on this feature Extending Copilot with CDK(https://github.com/aws/copilot-cli/issues/4208) and yaml patches (https://github.com/aws/copilot-cli/issues/4209). With these features, It will a lot easier to override the existing Copilot resources in this case IAM Roles with your own customized IAM roles. We will update once the feature is released.

As a WorkAround here is an example of creating an IAM role TaskRole for an ECS task until the CDK extensions is released.

  1. You need to delete the existing TaskRole for your service.
  2. Create an single IAM role per environment and use this IAM role for all the services with the following policy below.
    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "iam:*",
            "Resource": "*",
            "Effect": "Deny"
        },
        {
            "Condition": {
                "StringEquals": {
                    "iam:ResourceTag/copilot-application": "<Application-Name>",  <----- replace with application name
                    "iam:ResourceTag/copilot-environment": "<Environment-Name>" <---- replace with environment name
                }
            },
            "Action": "sts:AssumeRole",
            "Resource": [
                "arn:aws:iam::1111111111:role/*"
            ],
            "Effect": "Allow"
        },
        {
            "Action": [
                "ssmmessages:CreateControlChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenDataChannel"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "logs:CreateLogStream",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:PutLogEvents"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
    }
  3. Update the manifest file of all the services taskdef_overrides below. This will update the TaskRoleArn in the ECS TaskDefinition.

    taskdef_overrides:
    - path: "TaskRoleArn"
    value: "<ARN of IAM Role>".  <---- ARN of the customized IAM role.

    Let me know if this works for you!

zmad5306 commented 1 year ago

@KollaAdithya This worked for the TaskRole can the Task Execution Role be overridden in the same way?

KollaAdithya commented 1 year ago

Yes, Task Execution Role can also be overridden in the same way. Just update the manifest with taskdef_overrides below

taskdef_overrides:
  - path: "ExecutionRoleArn"
    value: "<ARN of IAM Role>".  <---- ARN of the customized TaskExecution IAM role.
zmad5306 commented 1 year ago

@KollaAdithya The work around described doesn't fix the Quota issue and has other breaking issues with jobs:

  1. The built in Task Role and Task Execution Role, dynamically created by Copilot, are still included in the Cloud Formation template which Copilot generates. Thus they are still created (and not used) and count towards the quota.
  2. Jobs create a StateMachineRole and a StateMachine permission, this permission references the build it roles in the iam:PassRole permission instead of the custom roles passed in the Task Def Override and renders jobs inoperable.

This seems like a very rough edge for any organization deploying to ECS using Copilot at any scale.