search

aws-ia / terraform-aws-eks-blueprints-addons

Terraform module which provisions addons on Amazon EKS clusters
https://aws-ia.github.io/terraform-aws-eks-blueprints-addons/main/
Apache License 2.0
269 stars 127 forks source link

EKS Add-ons v1.18 seems to not like my default tags #424

Open mmoreno43 opened 4 weeks ago

mmoreno43 commented 4 weeks ago

Description

Hello, the latest release (1.18.0) has broke my terraform. It seems as though it does not like the tags I have in place in my default tags.

Versions

Reproduction Code [Required]

main.tf

module "eks_blueprints_addons" {
  source = "aws-ia/eks-blueprints-addons/aws"

  cluster_name      = module.eks_dev.cluster_name #update eks module
  cluster_endpoint  = module.eks_dev.cluster_endpoint #update eks module
  cluster_version   = module.eks_dev.cluster_version #update eks module
  oidc_provider_arn = module.eks_dev.oidc_provider_arn #update eks module

  enable_cluster_autoscaler = true
  cluster_autoscaler = {
    values = [
    "${file("${path.module}/helm-charts/cluster-autoscaler.yaml")}"
    ]

    set = [{
        name = "fullnameOverride"
        value = "cluster-autoscaler"
    }] 
  }

  enable_external_dns = true
  external_dns = {
    values = [
    "${file("${path.module}/helm-charts/external-dns.yaml")}"
    ]
  }
  external_dns_route53_zone_arns = [
    "arn:aws:route53:::hostedzone/xxxxxxxxx", #update hosted zone ID
  ]

  enable_ingress_nginx = true
  ingress_nginx = {
    values = [
    "${file("${path.module}/helm-charts/ingress-nginx.yaml")}"
    ]

    set = [{
        name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-subnets" 
        value = "${data.terraform_remote_state.all.outputs.module_paas_subnet_01_id}\\,${data.terraform_remote_state.all.outputs.module_paas_subnet_02_id}\\,${data.terraform_remote_state.all.outputs.module_paas_subnet_03_id}\\,${data.terraform_remote_state.all.outputs.module_paas_subnet_04_id}"
        #update values to VPC ids
    }]
  }

  enable_aws_efs_csi_driver = true
  aws_efs_csi_driver = {
    set = [{
        name  = "useFips"   
        value = "true"
    }]

    set = [{
        name  = "controller.serviceAccount.create"  
        value = "true"
    }]

    set = [{
        name  = "controller.serviceAccount.name"    
        value = "efs-csi-controller-sa"
    }]

  }

  tags = {
  }

  enable_metrics_server = true
  metrics_server = {
    values = [
      "${file("${path.module}/helm-charts/metrics-server.yaml")}"      
    ]
    set = [
      {
        name  = "forceInstall" #make sure it is installed, even when terraform states marks as no change
        value = "true"
      },
      {
        name  = "apiService.create"
        value = "true"
      }
    ]
  }

}

providers.tf


provider "aws" {
    region  = "us-west-2"
    assume_role {
        role_arn = "arn:aws:iam::ENV_AWS_ACCOUNT_NUMBER:role/xxxx"
    }
    default_tags {
        tags = {
            "Business Unit"              = var.Business_Unit
            "Cost Center"                = var.Cost_Center
            "Technology Business Unit"   = var.Technology_Business_Unit
            "IT Owner"                   = var.IT_Owner
            "Ops Team"                   = var.Ops_Team
            "Product App Name"           = var.Product_App_Name
            "IT Support Team"            = var.IT_Support_Team
            "SNOW Asset ID"              = var.SNOW_Asset_ID
            "App Type"                   = var.App_Type
            "Platform Name"              = var.Platform_Name
            "Jenkins Org Pipeline"       = var.Jenkins_Org_Pipeline
            "CI Code Repository"         = var.CI_Code_Repository
            "CD Code Repository"         = var.CD_Code_Repository
            "Environment"                = "${var.Environment}"
            "Business Department"        = var.Business_Department
            "Secrets PF"                 = var.Secrets_PF
            "Break Glass PF"             = var.Break_Glass_PF
            "Logs PF"                    = "${var.Logs_PF}"
            "Security Scans"             = "${var.Security_Scans}"
            "Xmatters"                   = var.Xmatters
            "Documentation URL"          = var.Documentation_URL

      }
    }
}

terraform.tfvars


Business_Unit               = "TECH"
Cost_Center                 = "kjds98C-IT-djd-jsdls"
Technology_Business_Unit    = "Tech Tesh"
IT_Owner                    = "John Apple"
Ops_Team                    = "Test Tech"
Product_App_Name            = "TEST"
IT_Support_Team             = "TEST"
SNOW_Asset_ID               = ""
App_Type                    = ""
Platform_Name               = "TEST"
Jenkins_Org_Pipeline        = "https://jenkins.com"
CI_Code_Repository          = "https://github.com"
CD_Code_Repository          = "https://github.com"
Artifact_Package            = ""
Environment                 = "ENV_ENVIRONMENT_NAME"
Business_Department         = "TECH"
Secrets_PF                  = "Vault"
Break_Glass_PF              = "CyberArk"
Logs_PF                     = "Cloud Watch/Cloud Trail"
Security_Scans              = "Laceworks"
Xmatters                    = ""
Documentation_URL           = "https://confluence.com/wiki/spaces/TEST"

Steps to reproduce the behavior:

terraform init terraform plan terraform apply

Expected behavior

I expect that Terraform will successfully apply "module.eks_blueprints_addons.aws_cloudformation_stack.usage_telemetry[0]."

Actual behavior

Terraform fails during apply

Terminal Output Screenshot(s)

image

Additional context

We have been using this module on in our environment for a while across multiple clusters. Would it be possible to have a flag so that we can disable this if we don't want it so that we aren't pinned to version 1.17.0 and can receive future updates?

bryantbiggs commented 4 weeks ago

we can probably add something to paper over this but it looks like the issue is that you are providing tags without values 

SNOW_Asset_ID               = ""
App_Type                    = ""
bryantbiggs commented 4 weeks ago

actually, I take that back - we could do something if these values were passed directly to the module, but these are provided via the default_tags on the provider which we don't have access to those. you'll need to ensure you provide values for the associated keys supplied with tags

mmoreno43 commented 4 weeks ago

Could we make this a true/false input to deploy this or not? We create other AWS resources that are not affected by this but CloudFormation.

WarpRat commented 3 weeks ago

Or at least include a random string after the stack name. We invoke this module multiple times in our EKS module to time different groups of addons that need to be deployed to a cluster in a specific order, v1.18.0 breaks that for us with no way to override the stack name from matching the cluster name. We don't mind sharing telemetery but I don't think users only invoking this module once per cluster name is a safe assumption.

edit: Sorry, I guess I didn't read the initial issue closely enough. We're having a similar problem but with a name conflict on the telemetry cloudformation stack name. I'm happy to open a separate issue for that if it would be helpful.