Open pahud opened 7 months ago
For visibility, there’s a recent change that was released in 2.124.0 to only allow integ-runner
to run javascript
test file in experimental modules and aws-cdk-lib
, so all the new ts test file needs to be built through yarn build into js test file. This could be the reason that caused the failure.
@GavinZZ Thank you but I guess it might not be related.
This works for me with the following steps with cdk 2.125.0
// initiate a new project
$ cdk init -l typescript
$ mkdir test/integ
// edit test/integ/integ.main.ts and past all content mentioned above in the ts file and save.
$ yarn add @aws-cdk/integ-tests-alpha
$ yarn add @aws-cdk/integ-runner
$ npx integ-runner --directory test/integ
You see the error message
Verifying integration test snapshots...
NEW integ.main 1.999s
Snapshot Results:
Tests: 1 failed, 1 total
Failed: /Users/xxx/repos/repro5/test/integ/integ.main.ts
Error: Some tests failed!
To re-run failed tests run: integ-runner --update-on-failed
at main (/Users/xxx/repos/repro5/node_modules/@aws-cdk/integ-runner/lib/index.js:10424:13)
Now, run again with --update-on-failed
. It works with snapshots generated.
% npx integ-runner --directory test/integ --update-on-failed
Verifying integration test snapshots...
NEW integ.main 4.581s
Snapshot Results:
Tests: 1 failed, 1 total
Failed: /Users/xxx/repos/repro5/test/integ/integ.main.ts
Running integration tests for failed tests...
Running in parallel across regions: us-east-1, us-east-2, us-west-2
Running test /Users/xxx/repos/repro5/test/integ/integ.main.ts in us-east-1
SUCCESS integ.main-cdk-integ/DefaultTest 14.391s
NO ASSERTIONS
Test Results:
Tests: 1 passed, 1 total
However, if you initiate your project with projen
$ npx projen new awscdk-app-ts
edit .projenrc.js
const { awscdk } = require('projen');
const project = new awscdk.AwsCdkTypeScriptApp({
cdkVersion: '2.1.0',
defaultReleaseBranch: 'main',
name: 'repro5',
devDeps: [
'@aws-cdk/integ-tests-alpha',
'@aws-cdk/integ-runner',
],
});
project.synth();
$ npx projen
$ mkdir test/integ
// edit test/integ/integ.main.ts and save
$ npx integ-runner --directory test/integ // got the error
$ npx integ-runner --directory test/integ --update-on-failed
Then I get the error
% npx integ-runner --directory test/integ --update-on-failed
Verifying integration test snapshots...
NEW integ.main 1.754s
Snapshot Results:
Tests: 1 failed, 1 total
Running integration tests for failed tests...
Running in parallel across regions: us-east-1, us-east-2, us-west-2
Running test /Users/xxx/repos/repro5/test/integ/integ.main.ts in us-east-1
ERROR /Users/xxx/repos/repro5/test/integ/integ.main.ts (undefined/us-east-1) 1.645s
Error during integration test: Error: integ.main is a new test. Please use the IntegTest construct to configure the test
https://github.com/aws/aws-cdk/tree/main/packages/%40aws-cdk/integ-tests
Test Results:
Tests: 1 failed, 1 total
Error: Some integration tests failed!
at main (/Users/xxx/repos/repro5/node_modules/@aws-cdk/integ-runner/lib/index.js:10367:15)
This is a very confusing error. I have upgraded from 2.100 to 2.122 and got completely destroyed with the integ-runner.
I get the alpha status and all, but it'd be great to get some more context on these errors. These errors are not readable and do not point to any docs.
Error 1:
If your app has multiple stacks, specify which stack to select by adding this to your test source:
/// !cdk-integ STACK ...
That's just super weird and feels like a hack.
Error 2:
ERROR .../foo.itest.ts (undefined/us-east-1) 0.411s
Error during integration test: Error: foo.itest is a new test. Please use the IntegTest construct to configure the test
https://github.com/aws/aws-cdk/tree/main/packages/%40aws-cdk/integ-tests-alpha
u wot m8?
After a long search I found it:
You need to upgrade the "@aws-cdk/integ-runner" to a newer version. By default it picks something around 2.72 which does not work...
@hoegertn Thank you for your time. I will validate that again.
Describe the bug
integ-runner is complaining
while my test has been using
IntegTest
in the test.Expected Behavior
integ-runner should not complain the error
Current Behavior
It's complaining the error(see reproduction steps below)
Reproduction Steps
in test/integ/integ.main.ts
When I run
npx integ-runner --directory test/integ
, I gotAnd run it again with
--update-on-failed
I got
But I am actually using
IntegTest
.Possible Solution
No response
Additional Information/Context
integ-runner version "2.72.1" integ-tests-alpha version "2.124.0-alpha.0"
CDK CLI Version
2.124.0 (build 4b6724c)
Framework Version
No response
Node.js Version
v18.16.0
OS
Mac OS X
Language
TypeScript
Language Version
No response
Other information
No response