keikoproj / lifecycle-manager

Graceful AWS scaling event on Kubernetes using lifecycle hooks
Apache License 2.0
93 stars 28 forks source link

Abandoned instances don't drop active goroutines #34

Closed eytan-avisror closed 4 years ago

eytan-avisror commented 4 years ago

Is this a BUG REPORT or FEATURE REQUEST?: Bug

What happened: Related to #23 very similar to the above issue it seems that hooks that have timed out still have goroutines for alb deregister active.

In very large clusters with a large number of target groups, it was noticed that after an instance was abandoned there were still go routines that were going over the hundreds of target groups trying to deregister the already terminated instances.

What you expected to happen: We should probably kill all the associated goroutines once the instances is abandoned