aws-ia / terraform-aws-eks-blueprints

Configure and deploy complete EKS clusters.
https://aws-ia.github.io/terraform-aws-eks-blueprints/
Apache License 2.0
2.72k stars 1.43k forks source link

Unable to attach alb to nodes autoscaling group #854

Closed olibenni closed 2 years ago

olibenni commented 2 years ago

Description

Using aws_autoscaling_attachment to attach alb target group arn with out autoscaling group for the nodes in the eks only works on every other apply. The apply is successful but the attachment is removed on every other apply.

Versions

Reproduction Code [Required]

module "eks_blueprints" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.6.2"

  cluster_version    = "1.21"
  vpc_id             = var.vpc_id
  private_subnet_ids = var.subnet_ids
  cluster_name       = "somecluster-eks"

  self_managed_node_groups = {
    self_mg5 = {
      node_group_name    = "self_mg5"
      instance_type      = "m5.large"
      launch_template_os = "amazonlinux2eks"
      subnet_ids         = var.worker_subnet_ids
      public_ip          = true
      # This is an invalid argument at the moment
      # target_group_arns  = [var.aws_lb_target_group_arn]
  }
}

resource "aws_autoscaling_attachment" "asg_attachment_bar" {
  autoscaling_group_name = module.eks_blueprints.self_managed_node_group_autoscaling_groups[0]
  alb_target_group_arn   = var.aws_lb_target_group_arn
}

Expected behavior

Expect new nodes need be be part of our target group The autoscaling group in the eks cluster needs to be aware of the target group so that new nodes are part of it

Actual behavior

When we first apply, the attachment is made and things work fine. But in our subsequent apply, the eks_blueprint notices an attachment that isn't defined in the blueprint so it removes it.

askulkarni2 commented 2 years ago

@olibenni out of curiosity, what is your use-case to attach an ALB directly to the ASG this way?

olibenni commented 2 years ago

Good question, I wanted to add a loadbalancer for the ArgoCD server, exposing the ArgoCD web ui. Open to alternatives.

askulkarni2 commented 2 years ago

Gotcha. What you are trying to do will certainly work but its more common to use an Ingress configuration for such purposes. An Ingress controller such as AWS Load Balancer Controller makes it easier to manage Elastic Load Balancers for a Kubernetes cluster from within the cluster. Checkout our example which shows how you can configure a complete Ingress path with a combination of ingress-nginx, LBC, external-dns. You can repurpose this example for serving of the ArgoCD UI.

olibenni commented 2 years ago

Thanks for the tip! Will check this out better tomorrow (working on UTC time).

bryantbiggs commented 2 years ago

closed in #855