Reached 'Managed Policies Per Role' Limit

[blazinaj]

There is a hard limit on how many 'Managed Policies' can be attached to an IAM Role. The limit starts at 10 and can only be increased to 20.

Amplify CLI utilizes managed policies for the frontend 'auth' role, and adding new environment resources (API Models, Functions, etc), more policies must be generated and attached to the auth role.

We currently are at 19 policies and will soon hit the hard limit of 20, in our sandbox environments we have already been getting failed deployments when adding new resources.

What is our best recourse here? We use Amplify CLI in a production environment and have very active development going on.

[lazpavel]

Hello,

Thank you for raising the question. Currently Cognito can attach only one IAM role to an auth and unauth identity, which is a constraint and one role can have a limited set of policies. Can you provide the list of all managed policies tied to a single IAM role and then we can analyze it and see if we can optimize any of them?

[blazinaj]

What's the best way to get that to you privately? There are 19 managed policies that are full of account numbers and schema details

[lazpavel]

Hello @blazinaj, Please send them over to amplify-cli@amazon.com Thank you!

[blazinaj]

Sent them over, any workaround ideas would be greatly appreciated! This limit is putting us in a really bad spot, we've been developing using Amplify CLI for over 3 years now and are pretty heavily invested in the infrastructure here. Thank you for your prompt responses!

[josefaidt]

Hey @blazinaj :wave: apologies for the delay here! Are you still experiencing this?

[blazinaj]

Yes. We have been unable to add new schema models for a long time because of this.

Stumbled upon this other issue that relates very closely. We've found that it doesn't even look like we need or want these policies to be made, since we are only using the @iam directive to allow our Lambdas to interact with the data on the backend, and these policies are being generated for the frontend. But we only use Cognito auth from the frontend.

We can't figure out a way to get the CLI to not try and generate these policies. So every time we run a push operation, the entire pipeline fails when it attempts to generate the 21st policy

[josefaidt]

Hey @blazinaj apologies for the delay here, can you clarify how you've reached this limit? Is this after adding X amount of resources to an environment or via some other route?

[josefaidt]

Hey @blazinaj just wanted to follow up on this one and see if you are still experiencing this?

[blazinaj]

Hello, I apologize for the delay.

Yes we are still experiencing this. This does come when we add the @iam directive to too many @model definitions. It adds too many managed policies to the authRole.

We have to use the @iam directive so that our lambdas can access the data, we are using purely Cognito on the front end. We would like to just disable this policy generation entirely, since we don't need most of them and this bricks our entire CI pipeline.

[SaileshKumar]

Just ran into this as well. We have the same use case of using @iam so lambdas can access the data, and just hit the limit

[josefaidt]

Hey @blazinaj and @SaileshKumar :wave: apologies for the delay here! Are y'all still experiencing this? In an attempt to reproduce I created 20 models with model-level IAM auth rules, and was able to push successfully as the policies are constructed in such a way that we add the models to the same PolicyDocument and split accordingly: Do you have an example of an offending schema? Are you using an imported auth resource?

[SaileshKumar]

Yea - if I remember correctly I hit the 10 limit and then upgraded to 20, and haven't hit that one yet. Happy to email you a schema if that helps. Yes I am using an imported Cognito Pool

EDIT - I actually just hit the 20 limit haha

[josefaidt]

Hey @SaileshKumar thanks for confirming and sending the schema over! I was able to reproduce this with the following steps:

1. amplify init -y
2. amplify import auth > choose an existing auth resource
3. amplify add api > add Cognito, IAM as auth methods
4. create 20-30 models with IAM auth rules

   type Todo1 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
     name: String!
     description: String
   }
   type Todo2 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
     name: String!
     description: String
   }
   type Todo3 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
     name: String!
     description: String
   }
   type Todo4 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
     name: String!
     description: String
   }
   type Todo5 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
     name: String!
     description: String
   }
   type Todo6 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
     name: String!
     description: String
   }
   type Todo7 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo8 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo9 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo10 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo11 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo12 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo13 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo14 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo15 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo16 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo17 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo18 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo19 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo20 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo21 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo22 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo23 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo24 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo25 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo26 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo27 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo28 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo29 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }
   type Todo30 @model @auth(rules: [{ allow: private, provider: iam }]) {
     id: ID!
   }

5. push with amplify push -y
6. create another environment (this will then attempt to add a similar set of policies to the authrole) with amplify env add next
7. push with amplify push -y
8. observe error

🛑 The following resources failed to deploy:
Resource Name: AuthRolePolicy02A05DD8DC (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PoliciesPerRole: 10 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: cbbc05f9-1bc7-4abe-b6d7-d565bc632932; Proxy: null)

Resource Name: AuthRolePolicy049E37F651 (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PoliciesPerRole: 10 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: f4489a5a-005d-4070-ab14-210a992959ce; Proxy: null)

Resource Name: AuthRolePolicy03DA271F4C (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PoliciesPerRole: 10 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: ab2cdbff-0ec1-4997-9e6c-44d78d8558bb; Proxy: null)

Resource Name: AuthRolePolicy06CEB1E51C (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PoliciesPerRole: 10 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: b8ce824f-8655-421e-a06d-0a03fdac53ca; Proxy: null)

Resource Name: AuthRolePolicy01921FC820 (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PoliciesPerRole: 10 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: f796dca8-01dd-4827-aeae-fd32f91ec458; Proxy: null)

Resource Name: AuthRolePolicy0552AC47CD (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PoliciesPerRole: 10 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: 6784537a-9f63-47b8-b16c-0c8d31e271aa; Proxy: null)

Project ID: 907c4574be01ca01dbfe41a36a305b8b

Marking as a bug 🙂

[alharris-at]

Hi folks, a couple of ideas on how to work around here, and background on what is causing this issue:

Note: the root problem here is that when using IAM auth, the authRole (and potentially the unauthRole if enabled) will need to have a policy document created which grants access to all AppSync queries linked. I took a look at the code, and already grab all resources that require access, and split policy documents once they hit the size limit that IAM allows. A policy doc has a max payload of 6144 bytes.

So we split this into policies of size 6144, then attach each of those policies to the authRole (and optional unauthRole).

Some ideas to potentially shave resources in the policy off.

• Disable unnecessary queries/mutations/subscriptions on the API - https://docs.amplify.aws/cli/graphql/data-modeling/#disable-generated-queries-mutations-and-subscriptions if the Cx isn't using subscriptions for example, a reduction in overall resources attached to the role can be reduced by disabling generation of those, etc.
• Ensure they're not re-using auth resources across environments or apps. Using amplify import auth will result in multiple policies being attached to the same role, which may exacerbate this issue.
• Verify if IAM auth is required for all configured models, etc. If there are opportunities to rely on other auth mechanisms, that'll reduce the size of these policies.
• Consider if models can be combined. If using multiple model types w/ a hasOne relation, can those be consolidated, etc. A lower number of models will result in fewer resources attached to the iam role.

[SaileshKumar]

@josefaidt @alharris-at is there an ETA on when this could be resolved? Also re the second point:

"Ensure they're not re-using auth resources across environments or apps. Using amplify import auth will result in multiple policies being attached to the same role, which may exacerbate this issue."

Since we are using amplify import auth (but it's only being used for this environment, old one was deleted), is there a way to clean up the roles from that old environment?

Thanks!