search

amazon-archives / realworld-serverless-application

This project is inspired by the design and development of the AWS Serverless Application Repository - a production-grade AWS service. Learn how AWS built a production service using serverless technologies.
Apache License 2.0
515 stars 108 forks source link

Run integ tests against deployed test stack #53

Open jlhood opened 4 years ago

jlhood commented 4 years ago

Currently the backend integ tests are self-contained in that they stand up test stacks, run the integ tests against those stacks and then tear down the test stacks. This captures one type of integ testing we do in SAR, but not the other kind, which is to test against the test environment (beta) stacks deployed by the pipeline. This is causing some confusion so we should also add a pipeline step to run integ tests against the deployed instance of the application after pipeline deployment.

honglu commented 4 years ago

We considered this option but since there is no more stage after the Deploy stage, even if the integ tests fail, it does not really prevent anything or rollback the deployment in Deploy stage.

salmanap commented 4 years ago

Couldn't we create another stage in the pipeline? run the integ tests in that stage and then promote the changes to the final stage?

jlhood commented 4 years ago

Yeah I think it'd be better if the pipeline had a beta and prod stack with integ tests running against the beta stack prior to deployment to the prod stack.

honglu commented 4 years ago

Since we are using aws-sam-codepipeline-cd to setup CD pipeline, I created an issue there to address this issue