terraform-aws-modules / terraform-aws-eks

Terraform module to create Amazon Elastic Kubernetes (EKS) resources 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/eks/aws
Apache License 2.0
4.48k stars 4.09k forks source link

Reduce IAM Policy Size #3181

Open Giaco9NN opened 1 month ago

Giaco9NN commented 1 month ago

Is your request related to a problem? Please describe.

During the upgrade from v0.37.x to v.1.0.0, the IAM permissions required by karpenter changed, so the policy changed. By adding the parameter enable_v1_permissions = true, deploying the module throws the following error:

│ Error: updating IAM Policy (arn:aws-us-gov:iam::043318617890:policy/KarpenterController-20240212160616829700000005): operation error IAM: CreatePolicyVersion, https response error StatusCode: 409, RequestID: 5488c390-778f-4495-b6e0-1145b1564ca4, LimitExceeded: Cannot exceed quota for PolicySize: 6144

I'm deploying the module in the us-gov-east-1 region, and the cluster name is "gov-east-1-data-engineering-cluster-1-31" (which can't be changed). I'm attaching the generated policy, which is 6269 characters long.

terraform-aws-eks-3181.txt

Describe the solution you'd like.

I think there are a few alternatives without compromising the permissions We could reduce the sides. In total, they are 482 characters. Another option would be to split the policies into a couple of managed policies. The last option I can think of is to give the user the possibility to customise the policy by exposing the document as output and deciding whether to create it. In this way, the user can instrument the module to not create the managed policy but to customize the document and create the policy themselves.

In my opinion, considering it is not possible to customize the policy, we could just reduce the SIDS. I would like to help with the implementation of the solution you pick :)

Describe alternatives you've considered.

In my environments, I set the parameter enable_v1_permissions = false and created the policy by reducing SIDS sizes.

Additional context

Here is how I'm using the module:

module "karpenter" {
  source                          = "terraform-aws-modules/eks/aws//modules/karpenter"
  version                         = "20.26.0"
  cluster_name                    = module.eks.cluster_name
  enable_pod_identity             = false
  enable_irsa                     = true
  create_iam_role                 = true
  iam_role_name                   = "KarpenterController"
  iam_role_use_name_prefix        = false
  irsa_oidc_provider_arn          = module.eks.oidc_provider_arn
  ami_id_ssm_parameter_arns       = ["arn:*:ssm:*:*:parameter/aws/service/*"]
  irsa_namespace_service_accounts = ["kube-system:karpenter"]
  create_instance_profile         = false
  create_node_iam_role            = false
  create_access_entry             = false
  node_iam_role_arn               = local.workers_node_group_role_arn
  enable_v1_permissions           = false
  enable_spot_termination         = true
  tags                            = var.tags
}
bryantbiggs commented 1 month ago

these match the upstream policy

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days