DeleteConflict: Cannot delete entity [IAMRole], must delete policies first

[chr-b]

The error:

global - IAMRole - dummyrole0001 - [] - failed
ERRO[0033] DeleteConflict: Cannot delete entity, must delete policies first.
    status code: 409

Deletion of the resource IAMRole fails, even when also specifying IAMRolePolicyAttachment in the resource types.

My config:

regions:
- global

account-blocklist:
  - 123456789012

accounts:
  "987654321098":
    presets:
      - common

resource-types:
  includes:
    - IAMRolePolicyAttachment
    - IAMRole

presets:
  common:
    filters:
      IAMRole:
        - OrganizationAccountAccessRole
        - type: "glob"
          value: "stacksets-exec-*"
      IAMRolePolicyAttachment:
        - OrganizationAccountAccessRole
        - type: "glob"
          value: "stacksets-exec-*"

Output for aws-nuke run --config test.yaml --profile my-profile --assume-role-arn arnn:aws:iam::987654321098:role/OrganizationAccountAccessRole --assume-role-session-name nuker --no-prompt -no-dry-run -l debug:

aws-nuke - v3.2.1 - c6b7a1e7b6a4a5703a3d43c4d8a086ebc2e6eb95
Do you really want to nuke the account with the ID 987654321098 and the alias 'dummy'?
Waiting 3s before continuing.
global - IAMRole - AWSReservedSSO_AdministratorAccess_7bb55fc7e18652af - [] - cannot delete SSO roles
[...]
global - IAMRole - AWSServiceRoleForTrustedAdvisor - [] - cannot delete service roles
global - IAMRole - OrganizationAccountAccessRole - [] - filtered by config
global - IAMRole - dummyrole0001 - [] - would remove
global - IAMRole - dummyrole0002 - [] - would remove
global - IAMRole - stacksets-exec-cdc0d6bfed707f43676c0970e8145069 - [] - filtered by config
global - IAMRolePolicyAttachment - AWSReservedSSO_AdministratorAccess_7bb55fc7e18652af -> AdministratorAccess - [PolicyArn: "arn:aws:iam::aws:policy/AdministratorAccess", PolicyName: "AdministratorAccess", RoleCreateDate: "2021-10-14T13:51:03Z", RoleLastUsed: "2021-10-14T13:51:03Z", RoleName: "AWSReservedSSO_AdministratorAccess_7bb55fc7e18652af", RolePath: "/aws-reserved/sso.amazonaws.com/us-east-2/"] - cannot detach from SSO roles
[...]
global - IAMRolePolicyAttachment - AWSServiceRoleForTrustedAdvisor -> AWSTrustedAdvisorServiceRolePolicy - [PolicyArn: "arn:aws:iam::aws:policy/aws-service-role/AWSTrustedAdvisorServiceRolePolicy", PolicyName: "AWSTrustedAdvisorServiceRolePolicy", RoleCreateDate: "2020-11-18T14:42:59Z", RoleLastUsed: "2024-02-26T07:33:18Z", RoleName: "AWSServiceRoleForTrustedAdvisor", RolePath: "/aws-service-role/trustedadvisor.amazonaws.com/"] - cannot detach from service roles
global - IAMRolePolicyAttachment - stacksets-exec-cdc0d6bfed707f43676c0970e8145069 -> AdministratorAccess - [PolicyArn: "arn:aws:iam::aws:policy/AdministratorAccess", PolicyName: "AdministratorAccess", RoleCreateDate: "2023-09-06T07:47:55Z", RoleLastUsed: "2023-09-06T07:47:55Z", RoleName: "stacksets-exec-cdc0d6bfed707f43676c0970e8145069", RolePath: "/"] - filtered by config
Scan complete: 34 total, 2 nukeable, 32 filtered.

Do you really want to nuke the account with the ID 987654321098 and the alias 'dummy'?
Waiting 3s before continuing.
global - IAMRole - dummyrole0001 - [] - failed
global - IAMRole - dummyrole0002 - [] - failed

Removal requested: 0 waiting, 2 failed, 32 skipped, 0 finished

global - IAMRole - dummyrole0001 - [] - failed
global - IAMRole - dummyrole0002 - [] - failed

Removal requested: 0 waiting, 2 failed, 32 skipped, 0 finished

global - IAMRole - dummyrole0001 - [] - failed
global - IAMRole - dummyrole0002 - [] - failed

Removal requested: 0 waiting, 2 failed, 32 skipped, 0 finished

ERRO[0033] There are resources in failed state, but none are ready for deletion, anymore. 

global - IAMRole - dummyrole0001 - [] - failed
ERRO[0033] DeleteConflict: Cannot delete entity, must delete policies first.
    status code: 409, request id: dfd0fc9b-c733-4b8e-98da-e274e09c3224 
global - IAMRole - dummyrole0002 - [] - failed
ERRO[0033] DeleteConflict: Cannot delete entity, must delete policies first.
    status code: 409, request id: 79fa895a-7d63-4ae2-9612-9f8b0bd9d69f 
FATA[0033] failed

[ekristen]

My guess is there is either a Permissions Boundary set, or there are Permissions Policies attached that were accidentally omitted from the logs above?

Can you verify using the UI if either of those two are true, if it's Permissions Boundary, that's a recently known issue that hasn't had an issue opened against it yet.

[chr-b]

The role where the deletion failed has a policy of type "Customer inline" attached to it :(

Edit: for completeness, all the lines from the log with that role

global - IAMRole - dummyrole0001 - [] - would remove
[...]
Do you really want to nuke the account with the ID 987654321012 and the alias 'dummy'?
Waiting 3s before continuing.
global - IAMRole - dummyrole0001 - [] - failed

Removal requested: 0 waiting, 1 failed, 32 skipped, 0 finished

global - IAMRole - dummyrole0001 - [] - failed

Removal requested: 0 waiting, 1 failed, 32 skipped, 0 finished

global - IAMRole - dummyrole0001 - [] - failed

Removal requested: 0 waiting, 1 failed, 32 skipped, 0 finished

ERRO[0039] There are resources in failed state, but none are ready for deletion, anymore. 

global - IAMRole - dummyrole0001 - [] - failed
ERRO[0039] DeleteConflict: Cannot delete entity, must delete policies first.
    status code: 409, request id: dac8be7f-b138-4cac-8392-53040283b59d 

The attachment with the custom IAM policy is not detected.

[ekristen]

Thanks. That helped. It is indeed a separate API call for inline policies.

[ekristen]

ok you need to add IAMRolePolicy resource. It's a separate resource to handle inline policies.

[ekristen]

resource-types:
  includes:
    - IAMRolePolicyAttachment
    - IAMRole
    - IAMRolePolicy

[chr-b]

Thanks for the hint @ekristen , however this is more destructive then it should be. Log output:

global - IAMRolePolicy - OrganizationAccountAccessRole -> AdministratorAccess - [PolicyName: "AdministratorAccess", role:Path: "/", role:RoleID: "AROAQUR64X3RNKJAYUF76", role:RoleName: "OrganizationAccountAccessRole"] - would remove

This should not touch the OrganizationAccountAccessRole, but it does.

Config excerpt:

resource-types:
  includes:
    - IAMRolePolicyAttachment
    - IAMRolePolicy
    - IAMRole

presets:
  common:
    filters:
      IAMRole:
        - OrganizationAccountAccessRole
      IAMRolePolicyAttachment:
      - OrganizationAccountAccessRole
      IAMRolePolicy:
      - OrganizationAccountAccessRole

[ekristen]

This is where filters get a little complicated. Everything between the resource type and the [] is the "Legacy Resource Name", everything within the [] are properties.

When you do a bare filter like - OrganizationAccountAccessRole it's an exact match on the Legacy Resource Name, in this case that means OrganizationAccountAccessRole -> AdministratorAccess which it won't match. You'd need to add that whole string or add a filter type and use the operator contains for example. IAMRolePolicyAttachment is the same way, if you look at the output you previously provided it has <role> -> <policy>.

[chr-b]

Ah, thanks for the explanation.

Changing the filter to use property role:RoleName fixed the problem.