Netflix / asgard

[Asgard is deprecated at Netflix. We use Spinnaker ( www.spinnaker.io ).] Web interface for application deployments and cloud management in Amazon Web Services (AWS). Binary download: http://github.com/Netflix/asgard/releases
http://netflix.github.com/asgard
Apache License 2.0
2.24k stars 403 forks source link

Flaky test in DeploymentWorkflowSpec #642

Open holmesjr opened 9 years ago

holmesjr commented 9 years ago

The test "should not rollback if previous ASG has disappeared" in DeploymentWorkflowSpec seems to be flaky. Something in the interaction of the mock throwing the exception:

then: 1 * mockActivities.reasonAsgIsNotOperational(userContext, 'the_seaward-v003', 1) >> {
            throw new IllegalStateException('Something really went wrong!')
        }

and the code in DeploymentWorkflowImpl at scaleAsg():

DoTry<Void> asgIsOperational = doTry {
            retry(retryPolicy) {
                waitFor(activities.reasonAsgIsNotOperational(userContext, asgName, capacity)) { String reason ->
                    if (reason) {
                        throw new PushException(reason)
                    }
                    Promise.Void()
                }
            }
        }

causes the test to hang on some occasions at line 232:

waitFor(anyPromises(startupTimeout.result, asgIsOperational.result)) {...

Sometimes the test passes fine. We see this pass/fail behaviour in subsequent runs of the test.