integ-tests: add customer focused documentation on using integration tests

[diranged]

Describe the bug

We choose to not use the default bootstrap qualifier name (hnb659fds), and instead rename it default. For our CDK apps this works fine, as we can just update the cdk.json file and be done. However, we have noticed that the Integration Tests (integ-tests-alpha and integ-runner modules) do not honor either the cdk.json file OR passing in a custom Synthesizer option.

See Slack Thread: https://cdk-dev.slack.com/archives/C018XT6REKT/p1667598499102579

Expected Behavior

I would expect that both the cdk.json and the custom synthesizer would be honored, but neither seem to be.

Current Behavior

You seem to be stuck using the default bootstrap qualifier name, and that's a shame.

Reproduction Steps

Example Code:

const app = new App();
const stack = new Stack(app, "root", {
  env: env,
  synthesizer: new DefaultStackSynthesizer({
    qualifier: "default",
  }),
});
new NestedVpcStack(stack, "vpc", {});

new IntegTest(app, "IntegTest", {
  testCases: [stack],
});

Output:

✨  Synthesis time: 0.52s

IntegTest/DefaultTest/DeployAssert: building assets...

current credentials could not be used to assume 'arn:aws:iam::xxxxx:role/cdk-hnb659fds-deploy-role-xxxx-us-west-2', but are for the right account. Proceeding anyway.
root: building assets...

[0%] start: Building 38c0da6d553db6d234152b663483ada66ada1d1c171ca6453fc401569f821b28:xxxx-us-west-2

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.50.0

Framework Version

No response

Node.js Version

18.9.0

OS

Mac OSX

Language

Typescript

Language Version

typescript 4.8.4

Other information

No response

[corymhall]

@diranged IntegTest creates an assertions stack by default (which uses the default synthesizer). You can pass a custom stack instead. https://docs.aws.amazon.com/cdk/api/v2/docs/@aws-cdk_integ-tests-alpha.IntegTest.html#assertionstack

const assertionStack = new Stack(app, "root", {
  env: env,
  synthesizer: new DefaultStackSynthesizer({
    qualifier: "default",
  }),
});

new IntegTest(app, "IntegTest", {
  testCases: [stack],
  assertionsStack,
});

[diranged]

Thank you @corymhall, this does fix the issue. So the idea here is that the synthesizer on my stack was fine, but that the default-created assertions stack also needed that setting. Technically you can close this issue - but I will note that the integration tests are a huge mystery so far to me. The documentation is really sparse, the examples are few and far between.

The only working (for me) example I have found is https://github.com/cdklabs/cdk-enterprise-iac/tree/main/test ... and while it does let me run the integration tests, its not integrated into jest so I can't run selective tests.

[mrgrain]

Yes @diranged I agree. We should at least add this to the docs. Ideally some magic would pick up a customer synthesizer from the stack under test. But this might actually be even more confusing.

[corymhall]

@diranged thanks for the feedback! I'll leave this open to track adding better documentation. I think initially the documentation was very focused on using it within this repo, but we need to add better documentation that is focused on customers using this for their own projects.