hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.61k stars 9k forks source link

[Bug]: aws_eks_addon cannot remove value for service_account_role_arn #30645

Open jgoldschrafe opened 1 year ago

jgoldschrafe commented 1 year ago

Terraform Core Version

1.3.8

AWS Provider Version

4.61.0

Affected Resource(s)

Expected Behavior

The EKS addon should be reconfigured to use the node's credentials instead of the previously-configured IRSA role ARN.

Actual Behavior

The provider emits a fatal validation error that appears to originate from the AWS SDK for Go.

Relevant Error/Panic Output Snippet

╷
│ Error: error updating EKS Add-On (my-cluster:vpc-cni): InvalidParameter: 1 validation error(s) found.
│ - minimum field size of 1, UpdateAddonInput.ServiceAccountRoleArn.
│ 
│ 
│   with module.eks.module.eks.aws_eks_addon.this["vpc-cni"],
│   on .terraform/modules/eks.eks/main.tf line 382, in resource "aws_eks_addon" "this":
│  382: resource "aws_eks_addon" "this" {
│ 
╵

Terraform Configuration Files

resource "aws_eks_addon" "vpc_cni" {
  cluster_name      = aws_eks_cluster.cluster.name
  addon_name        = "vpc-cni"
  addon_version     = "v1.12.6-eksbuild.1"
  resolve_conflicts = "PRESERVE"
}

Steps to Reproduce

  1. Apply the above Terraform against an EKS cluster.
  2. Assign a service account to the addon configuration using something like the following: aws eks update-addon --cluster-name my-cluster --addon-name vpc-cni --addon-version v1.12.6-eksbuild.1 --service-account-role-arn arn:aws:iam::111122223333:role/MyVPCCNIRole --resolve-conflicts PRESERVE
  3. Attempt to apply the above Terraform again to remove the service account configuration.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 1 year ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

mattburgess commented 1 year ago

@jgoldschrafe - I took a look at fixing this. The good news is I can reproduce the problem in a test case so it doesn't actually matter if the IRSA is added behind Terraform's back or not.

The bad news is that I haven't yet figured out a way to remove the IRSA from the add-on. It looks like you can't even do it from the AWS console; once I add an IRSA then the "Edit" screen for the addon will allow me to change the role to another IAM-provided one but I can't actually ask it to revert back to a none-IAM role.

If you've found a way to do this in the console or via the aws or eksctl CLI tools then I might be able to map that to the underlying API calls we need to make. But I suspect you might have to take this issue up with AWS directly to see if this is, or can be, supported. Raising it at https://github.com/aws/containers-roadmap/issues might be a good first step; I had a quick search through the existing issues, both open and closed, and couldn't immediately see anything relevant.

scott2449 commented 1 year ago

This happens for us even when using service_account_role_arn.. the first time we add/change service_account_role_arn to the resource it applies the annotation. Then if you run it again it will remove the arn annotation from the service account.

venkatamutyala commented 1 month ago

Having this issue with coredns. we mistakenly added a service_account_role_arn to our coredns add_on resource and now we cannot remove it.