When the CLI deploys a stack as part of an integration test, automatically deploy the stack to multiple environments within a single CLI execution
Use Case
A single CLI execution can deploy multiple stacks to multiple environments. This opens the door for bugs that arise from code that doesn't take into account that each environment may contain information that affects the deployment of the stack.
For example, code that uses any bootstrap stack resource must not assume that the resource is the same in all environments.
Automatically deploying stacks to multiple environments will catch these bugs without needing to explicitly write a test for it.
Proposed Solution
There are several technical approaches we can take:
Provide a base stack to be used in tests which duplicates itself across multiple environments.
Mutate the assembly at runtime.
...
For simplicity from an auth perspective, we can narrow this down to just multi-region tests and not deal with multi account.
Regardless though, since these tests require multiple environments, we'll need to make a change to our region allocator. We might be able to get away with just acquiring an additional region lease here:
But since we are planning a revamp to our environment allocation service, its likely better to just hold off until we implement it - taking into account requirements surfaced by this use-case.
Other Information
Some peripheral considerations:
We need to provide a way for tests to opt out of this behavior (there may be good reasons for it).
Do we add this as a permutation of the test? or instead?
Acknowledgements
[x] I may be able to implement this feature request
mrgrain
commented
1 week ago
Describe the feature
When the CLI deploys a stack as part of an integration test, automatically deploy the stack to multiple environments within a single CLI execution
Use Case
A single CLI execution can deploy multiple stacks to multiple environments. This opens the door for bugs that arise from code that doesn't take into account that each environment may contain information that affects the deployment of the stack.
For example, code that uses any bootstrap stack resource must not assume that the resource is the same in all environments.
Automatically deploying stacks to multiple environments will catch these bugs without needing to explicitly write a test for it.
Proposed Solution
There are several technical approaches we can take:
For simplicity from an auth perspective, we can narrow this down to just multi-region tests and not deal with multi account.
Regardless though, since these tests require multiple environments, we'll need to make a change to our region allocator. We might be able to get away with just acquiring an additional region lease here:
https://github.com/aws/aws-cdk/blob/33872f38ca4ef33e05d7f01ab36c192a30ab9137/packages/%40aws-cdk-testing/cli-integ/lib/resource-pool.ts#L69-L76
But since we are planning a revamp to our environment allocation service, its likely better to just hold off until we implement it - taking into account requirements surfaced by this use-case.
Other Information
Some peripheral considerations:
Acknowledgements
CDK version used
ALL
Environment details (OS name and version, etc.)
ALL