aws-ia / terraform-aws-eks-blueprints-addons

Terraform module which provisions addons on Amazon EKS clusters
https://aws-ia.github.io/terraform-aws-eks-blueprints-addons/main/
Apache License 2.0
272 stars 127 forks source link

Trying to override image.repository inside of the AWS LoadBalancer Controller Helm Chart #188

Closed awserinkolp closed 1 year ago

awserinkolp commented 1 year ago

Please describe your question here

I am trying to utilize a private ECR repository for the aws-load-balancer-controller add-on container. Per the documentation, I tried using aws_load_balancer_controller_helm_config but Terraform 1.4.x says that it did not expect the block there. I have the following code "working" with Terraform, and it even shows the metadata override, but nothing changes on the EKS cluster, it still tries to pull from public ecr. Can anyone provide guidance and/or sample code? Many thanks in advance.

` module "eks_blueprints_addons" { source = "aws-ia/eks-blueprints-addons/aws"

cluster_name = module.eks.cluster_name cluster_endpoint = module.eks.cluster_endpoint cluster_version = module.eks.cluster_version oidc_provider_arn = module.eks.oidc_provider_arn

eks_addons = { aws-ebs-csi-driver = { most_recent = true } coredns = { most_recent = true } vpc-cni = { most_recent = true } kube-proxy = { most_recent = true } }

enable_aws_load_balancer_controller = true

aws_load_balancer_controller = { set = [ { name = "image.repository" value = "my.private.repo/eks/aws-load-balancer-controller" } ] } `

Provide a link to the example/module related to the question

https://aws-ia.github.io/terraform-aws-eks-blueprints/v4.32.1/add-ons/aws-load-balancer-controller/

awserinkolp commented 1 year ago

Update: If you specify it at the time the add-on is loaded into Kubernetes, it works. If you have already deployed the cluster add-on and then want to change it, Terraform/Helm "succeeds" on the apply but it doesn't reflect in the pod. Is this a bug?

askulkarni2 commented 1 year ago

Hi @awserinkolp thanks for the issue. I am not able to reproduce the issue. I do the see a redeployment being attempted (you can ignore the ErrImgPull) in the background.

aws-load-balancer-controller-7585d98bf8-t8ssc                1/1     Running        1 (28h ago)   34h
aws-load-balancer-controller-7585d98bf8-wz8sj                1/1     Running        1 (28h ago)   34h
aws-load-balancer-controller-c68466d9c-8t8sf                 0/1     ErrImagePull   0             107s

I did notice it takes a couple of seconds for the redeployment to begin. By default we set the wait to false. You may want to set it to true to block on it.

awserinkolp commented 1 year ago

@askulkarni2 Thank you so much for trying to reproduce! I'm not sure where I'm going wrong but we have a working path forward so all good.