API Lambda canary deployment - Githubissues

cds-snc / notification-planning

Project planning for GC Notify Team

4 stars 0 forks source link

API Lambda canary deployment #1078

Open sastels opened 1 year ago

sastels commented 1 year ago

Description

As a Notify developer, I need to be able to have confidence that releases do not break Notify.

Currently if for some reason the api lambda image crashes on startup, it will still be rolled out to staging / production, breaking Notify.
In kubernetes, if a pod won't boot the system keeps the old ones running so that Notify stays up.
We should have the same deployment process with the api lambda (or any other processes we move out of k8s).

This seems to be possible. See, for example

https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/perform-a-canary-based-deployment-using-the-blue-green-strategy-and-aws-lambda.html

https://ben11kehoe.medium.com/some-quick-thoughts-on-blue-green-deployment-for-lambda-with-cloudformation-ac66797984f

https://docs.aws.amazon.com/apigateway/latest/developerguide/canary-release.html

Acceptance Criteria** (Definition of done)

[ ] Broken lambdas do not deploy

QA Steps

[ ] Break the lambda and try to deploy (prefereably to staging)

yaelberger-commits commented 3 months ago

@sastels should this be a Notify Core task? Has this already been tackled? cc @jimleroyer