Describe the bug
We're currently upgrading our alb ingress controller to v2.4.4 from v1.2.0-alpha.1 in preparation for a kubernetes upgrade.
Hundreds of DescribeTargetHealth, DescribeTargetGroups, DescribeTags and RegisterTargets events are making us hit the API rate limits impacting all clusters - even the ones not upgraded to the new controller version.
How can I fix this issue? I've asked for an API rate limit increase but that's just a band-aid for this issue.
We're having issue creating and destroying ingresses because we hit the limits even when destroying the load balancers, leaving a lot of Terminating resources in our clusters.
Environment
AWS Load Balancer controller version v2.4.4
Kubernetes version v1.21.14
Using EKS (yes/no), if so version? yes
Additional Context:
We use these clusters to provision whole sets of applications in isolated environments for testing purposes, so we create and destroy dozens of environments per day, and each environment has between 20 and 30 ingress resources.
@lmbaschiera The fix has been shipped with the v2.5.2 release, I'm closing this issue as of now. Please feel free to reach out if you're still having this issue.
Describe the bug We're currently upgrading our alb ingress controller to v2.4.4 from v1.2.0-alpha.1 in preparation for a kubernetes upgrade. Hundreds of
DescribeTargetHealth
,DescribeTargetGroups
,DescribeTags
andRegisterTargets
events are making us hit the API rate limits impacting all clusters - even the ones not upgraded to the new controller version.How can I fix this issue? I've asked for an API rate limit increase but that's just a band-aid for this issue.
{"level":"error","ts":1682003556.0227876,"logger":"controller-runtime.manager.controller.ingress","msg":"Reconciler error","name":"release-8611","namespace":"","error":"Throttling: Rate exceeded\n\tstatus code: 400, request id: a450ce32-cf10-48a4-9736-aa5de377aa93"}
We're having issue creating and destroying ingresses because we hit the limits even when destroying the load balancers, leaving a lot of
Terminating
resources in our clusters.Environment
Additional Context: We use these clusters to provision whole sets of applications in isolated environments for testing purposes, so we create and destroy dozens of environments per day, and each environment has between 20 and 30 ingress resources.