terraform-aws-modules / terraform-aws-eks

Terraform module to create Amazon Elastic Kubernetes (EKS) resources 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/eks/aws
Apache License 2.0
4.38k stars 4.04k forks source link

Karpenter module terraform destroy running into: DeleteConflict: Cannot delete a policy attached to entities #3132

Open k24dizzle opened 3 weeks ago

k24dizzle commented 3 weeks ago

I'm running into the following error message when I try to terraform destroy resources created by the karpenter module.

Error: deleting IAM Policy (arn:aws:iam::1234567890:policy/karpenter-controller-policy): operation error IAM: DeletePolicy, https response error StatusCode: 409 ...  DeleteConflict: Cannot delete a policy attached to entities.

This is how I'm using the module

module "karpenter" {
  # https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/modules/karpenter/
  source  = "terraform-aws-modules/eks/aws//modules/karpenter"
  version = "20.24.0"

  cluster_name               = var.cluster_name
  create_access_entry        = false
  create_node_iam_role       = false
  enable_irsa                = true
  enable_v1_permissions      = true
  iam_policy_use_name_prefix = false
  iam_policy_name            = var.controller_policy_name # karpenter-controller-policy
  iam_role_name              = var.controller_role_name # karpenter-controller-role
  iam_role_use_name_prefix   = false
  irsa_namespace_service_accounts = [
    "${var.namespace}:${var.controller_service_account_name}",
  ]
  irsa_oidc_provider_arn = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${local.oidc_issuer_url}"
  node_iam_role_arn      = data.aws_iam_role.node_role.arn
  queue_name             = var.interruption_queue_name
  tags                   = var.tags
}

I'm curious if anyone else has run into this

When I look at what entities are attached to the policy, I see the controller role (iam_role_name)

% aws iam list-entities-for-policy --policy-arn arn:aws:iam::1234567890:policy/karpenter-controller-policy

{
    "PolicyGroups": [],
    "PolicyUsers": [],
    "PolicyRoles": [
        {
            "RoleName": "karpenter-controller-role",
            "RoleId": "redacted"
        }
    ]
}

I thought something like this would've helped, but it didn't when I tested it out.

k24dizzle commented 3 weeks ago

I think this issue might be related (using a count in aws_iam_role_policy_attachment and aws_iam_policy)