search

kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.94k stars 1.46k forks source link

unable to initialize AWS cloud - controller pod crash #3937

Closed adrianmiron closed 3 hours ago

adrianmiron commented 4 hours ago

Started getting the following error on pod startup 1h ago , on multiple clusters ( no update done to the app, EKS 1.30 , chart version 1.9.1 ). I thought it was something related to a single node, it was not....deleting a running ALB ingress pod and the replacement gets the error ( replaced nodes to make sure that is not it )

The controllers were not affected by any updates, no nodes were replaced, no updates on the clusters themselves. What is weird is that on a cluster in Singapore region it did replicate the issue

{"level":"error","ts":"2024-11-07T09:15:49Z","logger":"setup","msg":"unable to initialize AWS cloud","error":"failed to get VPC ID: failed to fetch VPC ID from instance metadata: error in fetching vpc id through ec2 metadata: get mac metadata: operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}

adrianmiron commented 3 hours ago

Nevermind....karpenter CRD updates changed the default for --http-put-response-hop-limit https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.7/deploy/installation/#using-the-amazon-ec2-instance-metadata-server-version-2-imdsv2