aws-ia / terraform-aws-eks-blueprints

Configure and deploy complete EKS clusters.
https://aws-ia.github.io/terraform-aws-eks-blueprints/
Apache License 2.0
2.73k stars 1.43k forks source link

addon: aws_node_termination_handler error when iterating over empty tuple #1537

Closed zack-is-cool closed 1 year ago

zack-is-cool commented 1 year ago

Description

We're currently using blueprints specifically for modules/kubernetes-addons.

I'm getting this error with enable_aws_node_termination_handler set to true

│ Error: Invalid index
│ 
│   on .terraform/modules/eks.eks_blueprints_kubernetes_addons/modules/kubernetes-addons/locals.tf line 14, in locals:
│   14:     awsNodeTerminationHandler = var.enable_aws_node_termination_handler ? module.aws_node_termination_handler[0].argocd_gitops_config : null
│     ├────────────────
│     │ module.aws_node_termination_handler is empty tuple
│ 

I am not defining manage_via_gitops, so it's defaulting to false via the variable block. There is an output tied to this that returns null:

# terraform-aws-eks-blueprints/modules/kubernetes-addons/aws-node-termination-handler/variables.tf
#...
variable "manage_via_gitops" {
  description = "Determines if the add-on should be managed via GitOps."
  type        = bool
  default     = false
}

# terraform-aws-eks-blueprints/modules/kubernetes-addons/aws-node-termination-handler/outputs.tf
#...
output "argocd_gitops_config" {
  description = "Configuration used for managing the add-on with ArgoCD"
  value       = var.manage_via_gitops ? local.argocd_gitops_config : null
}

Blueprints references this output in terraform-aws-eks-blueprints/locals.tf (here) without a try block and tries iterating over a null:

# terraform-aws-eks-blueprints/modules/kubernetes-addons/locals.tf
#...
  argocd_addon_config = {
    agones                    = var.enable_agones ? module.agones[0].argocd_gitops_config : null
    awsEfsCsiDriver           = var.enable_aws_efs_csi_driver ? module.aws_efs_csi_driver[0].argocd_gitops_config : null
    awsFSxCsiDriver           = var.enable_aws_fsx_csi_driver ? module.aws_fsx_csi_driver[0].argocd_gitops_config : null
    awsForFluentBit           = var.enable_aws_for_fluentbit ? module.aws_for_fluent_bit[0].argocd_gitops_config : null
    awsLoadBalancerController = var.enable_aws_load_balancer_controller ? module.aws_load_balancer_controller[0].argocd_gitops_config : null
    awsNodeTerminationHandler = var.enable_aws_node_termination_handler ? module.aws_node_termination_handler[0].argocd_gitops_config : null
    certManager               = var.enable_cert_manager ? module.cert_manager[0].argocd_gitops_config : null
    clusterAutoscaler         = var.enable_cluster_autoscaler ? module.cluster_autoscaler[0].argocd_gitops_config : null
    corednsAutoscaler         = var.enable_amazon_eks_coredns && var.enable_coredns_autoscaler && length(var.coredns_autoscaler_helm_config) > 0 ? module.coredns_autoscaler[0].argocd_gitops_config : null
    datadogOperator           = var.enable_datadog_operator ? module.datadog_operator[0].argocd_gitops_config : null
    grafana                   = var.enable_grafana ? module.grafana[0].argocd_gitops_config : null
    ingressNginx              = var.enable_ingress_nginx ? module.ingress_nginx[0].argocd_gitops_config : null
    keda                      = var.enable_keda ? module.keda[0].argocd_gitops_config : null
    metricsServer             = var.enable_metrics_server ? module.metrics_server[0].argocd_gitops_config : null
    ondat                     = var.enable_ondat ? module.ondat[0].argocd_gitops_config : null
    prometheus                = var.enable_prometheus ? module.prometheus[0].argocd_gitops_config : null
    sparkHistoryServer        = var.enable_spark_history_server ? module.spark_history_server[0].argocd_gitops_config : null
    sparkOperator             = var.enable_spark_k8s_operator ? module.spark_k8s_operator[0].argocd_gitops_config : null
    tetrateIstio              = var.enable_tetrate_istio ? module.tetrate_istio[0].argocd_gitops_config : null
    traefik                   = var.enable_traefik ? module.traefik[0].argocd_gitops_config : null
    vault                     = var.enable_vault ? module.vault[0].argocd_gitops_config : null
    vpa                       = var.enable_vpa ? module.vpa[0].argocd_gitops_config : null
    yunikorn                  = var.enable_yunikorn ? module.yunikorn[0].argocd_gitops_config : null
    argoRollouts              = var.enable_argo_rollouts ? module.argo_rollouts[0].argocd_gitops_config : null
    argoWorkflows             = var.enable_argo_workflows ? module.argo_workflows[0].argocd_gitops_config : null
    karpenter                 = var.enable_karpenter ? module.karpenter[0].argocd_gitops_config : null
    kubernetesDashboard       = var.enable_kubernetes_dashboard ? module.kubernetes_dashboard[0].argocd_gitops_config : null
    kubePrometheusStack       = var.enable_kube_prometheus_stack ? module.kube_prometheus_stack[0].argocd_gitops_config : null
    awsCloudWatchMetrics      = var.enable_aws_cloudwatch_metrics ? module.aws_cloudwatch_metrics[0].argocd_gitops_config : null
    externalDns               = var.enable_external_dns ? module.external_dns[0].argocd_gitops_config : null
    externalSecrets           = var.enable_external_secrets ? module.external_secrets[0].argocd_gitops_config : null
    velero                    = var.enable_velero ? module.velero[0].argocd_gitops_config : null
    promtail                  = var.enable_promtail ? module.promtail[0].argocd_gitops_config : null
    calico                    = var.enable_calico ? module.calico[0].argocd_gitops_config : null
    kubecost                  = var.enable_kubecost ? module.kubecost[0].argocd_gitops_config : null
    strimziKafkaOperator      = var.enable_strimzi_kafka_operator ? module.strimzi_kafka_operator[0].argocd_gitops_config : null
    smb_csi_driver            = var.enable_smb_csi_driver ? module.smb_csi_driver[0].argocd_gitops_config : null
    chaos_mesh                = var.enable_chaos_mesh ? module.chaos_mesh[0].argocd_gitops_config : null
    cilium                    = var.enable_cilium ? module.cilium[0].argocd_gitops_config : null
    gatekeeper                = var.enable_gatekeeper ? module.gatekeeper[0].argocd_gitops_config : null
    kyverno                   = var.enable_kyverno ? { enable = true } : null
    kyverno_policies          = var.enable_kyverno ? { enable = true } : null
    kyverno_policy_reporter   = var.enable_kyverno ? { enable = true } : null
    nvidiaDevicePlugin        = var.enable_nvidia_device_plugin ? module.nvidia_device_plugin[0].argocd_gitops_config : null
    consul                    = var.enable_consul ? module.consul[0].argocd_gitops_config : null
    thanos                    = var.enable_thanos ? module.thanos[0].argocd_gitops_config : null
    kubeStateMetrics          = var.enable_kube_state_metrics ? module.kube_state_metrics[0].argocd_gitops_config : null
  }

I think an easy solution would be to wrap all of these argocd_addon_config in terraform-aws-eks-blueprints/modules/kubernetes-addons/locals.tf configs in try() functions and defaulting to null?

Versions

Your version of Terraform is out of date! The latest version is 1.4.4. You can update by downloading from https://www.terraform.io/downloads.html


## Reproduction Code [Required]

<!-- REQUIRED -->

Steps to reproduce the behavior:

We're referencing blueprints like this:
```terraform
module "eks_blueprints_kubernetes_addons" {
  source = "git::https://github.com/aws-ia/terraform-aws-eks-blueprints.git//modules/kubernetes-addons?ref=v4.27.0"

  eks_cluster_id           = module.aws_eks.cluster_name
  eks_cluster_endpoint     = module.aws_eks.cluster_endpoint
  eks_oidc_provider        = module.aws_eks.oidc_provider
  eks_cluster_version      = module.aws_eks.cluster_version
  auto_scaling_group_names = concat(lookup(module.aws_eks.self_managed_node_groups, "autoscaling_group_name", []), lookup(module.aws_eks.eks_managed_node_groups, "node_group_autoscaling_group_names", []))

  # blueprints addons

  # EKS CoreDNS
  enable_amazon_eks_coredns = var.enable_amazon_eks_coredns
  amazon_eks_coredns_config = var.amazon_eks_coredns_config

  # EKS kube-proxy
  enable_amazon_eks_kube_proxy = var.enable_amazon_eks_kube_proxy
  amazon_eks_kube_proxy_config = var.amazon_eks_kube_proxy_config

  # EKS EBS CSI Driver
  enable_amazon_eks_aws_ebs_csi_driver = var.enable_amazon_eks_aws_ebs_csi_driver
  amazon_eks_aws_ebs_csi_driver_config = var.amazon_eks_aws_ebs_csi_driver_config

  # EKS Metrics Server
  enable_metrics_server      = var.enable_metrics_server
  metrics_server_helm_config = var.metrics_server_helm_config

  # EKS AWS node termination handler
  enable_aws_node_termination_handler      = var.enable_aws_node_termination_handler
  aws_node_termination_handler_helm_config = var.aws_node_termination_handler_helm_config

  # EKS Cluster Autoscaler
  enable_cluster_autoscaler      = var.enable_cluster_autoscaler
  cluster_autoscaler_helm_config = var.cluster_autoscaler_helm_config

  # Calico
  enable_calico      = var.enable_calico
  calico_helm_config = var.calico_helm_config
}

enable_aws_node_termination_handler- is set to true aws_node_termination_handler_helm_config - null atm

We don't want to use argo

Expected behaviour

Handle nulls safely

Actual behaviour

╷
│ Error: Invalid index
│ 
│   on .terraform/modules/eks.eks_blueprints_kubernetes_addons/modules/kubernetes-addons/locals.tf line 14, in locals:
│   14:     awsNodeTerminationHandler = var.enable_aws_node_termination_handler ? module.aws_node_termination_handler[0].argocd_gitops_config : null
│     ├────────────────
│     │ module.aws_node_termination_handler is empty tuple
│ 
│ The given key does not identify an element in this collection value: the collection has no elements.
╵
zack-is-cool commented 1 year ago

I'm looking at this closer double checking that it's not an issue with input causing the count on aws_node_termination_handler to be 0.

# inputs when calling module...
auto_scaling_group_names = concat(lookup(module.aws_eks.self_managed_node_groups, "autoscaling_group_name", []), lookup(module.aws_eks.eks_managed_node_groups, "node_group_autoscaling_group_names", []))

# terraform-aws-eks-blueprints/modules/kubernetes-addons/main.tf
#...
module "aws_node_termination_handler" {
  count                   = var.enable_aws_node_termination_handler && (length(var.auto_scaling_group_names) > 0 || var.enable_karpenter) ? 1 : 0
  source                  = "./aws-node-termination-handler"
  helm_config             = var.aws_node_termination_handler_helm_config
  manage_via_gitops       = var.argocd_manage_add_ons
  irsa_policies           = var.aws_node_termination_handler_irsa_policies
  autoscaling_group_names = var.auto_scaling_group_names
  addon_context           = local.addon_context
}

edit: Yeah I just need to use eks_managed_node_groups_autoscaling_group_names and self_managed_node_groups_autoscaling_group_names as my inputs.