search

spinnaker-plugins / aws-lambda-deployment-plugin-spinnaker

Spinnaker plugin to support deployment of AWS Lambda functions via Spinnaker pipelines
Apache License 2.0
23 stars 22 forks source link

feat: add the retry behaviour on 409 status code for all change requests #114

Closed armory-abedonik closed 2 years ago

armory-abedonik commented 2 years ago

Short summary

  1. Customer's pipeline(s) fails on lambda deployment steps. (with 409 status code)
  2. 409 status code returns when a stage (Lambda Deployment) is trying to update a resource (Lambda) which is already updating by a different stage, pipeline or even manually.
  3. Attempt to figure out what resource (or a person) updates the lambda in the same time with the pipeline wasn't successful.
  4. Attempt to find any code bugs what can cause 409 status code in AWS Lambda Plugin wasn't successful.

Conclusion Relaying on the findings above, I would suggest to introduce the retry behaviour on 409 status code for all change requests in AWS Lambda Plugin. By default, the retry feature will do 5 attempts over 30 seconds delay when 409 status code is returned. This approach will not solve this problem, but it will minimize the number of failures.