search

zalando-incubator / kube-ingress-aws-controller

Configures AWS Load Balancers according to Kubernetes Ingress resources
MIT License
375 stars 83 forks source link

Add test case for stack in RollbackInProgress state #667

Closed MustafaSaber closed 6 months ago

MustafaSaber commented 7 months ago

If a CloudFormation stack is in ROLLBACK_IN_PROGRESS it returns no output, hence no target groups. While CloudFormation is working to get the stack back the controller may send traffic to LBs without TG.

This PR is trying to replicate this behaviour, current state means that we also collect stacks in this state which we shouldn't do.

szuecs commented 7 months ago

You test the current state as far as I understand. If there is an error we will get into ROLLBACK state, but do you get from that test? I also don't get your description:

Trying to replicate detached target groups

How does your PR achieve this? I don't see it

I think what we want is to test for example a known wrong state (from our current alerting for example wrong WAF association) and then how we can come out of this state without delete/recreate (causes downtime), but maybe:

  1. create new stack
  2. delete broken stack

The order is important and delete needs to be postponed until some time in the future as we normally do. Of course this requires a much bigger test, not sure if that's easily possible.

szuecs commented 6 months ago

:+1: