Command npm run e2e --headless that runs cypress e2e tests doesn't return non-zero exit code on test failure. If wecompare it to protractor then it returns exit code 1 in case of e2e test failure.
Current Behavior
By running npm run e2e && echo $? the command returns exit code 0 in case of failed e2e test.
By running npm run e2e --headless on a test with failure it will return non-zero exit code.
This exit code information can then be used to determine the e2e test phase result on CI, if it was success or not.
Steps to Reproduce
npx create-nx-workspace --preset=angular
edit the pre-existing e2e test in a way it will fail
npm run e2e --headless && echo $?
exit code will be 0 (regardless of success or failure)
Failure Logs
Exit code after a unsuccessful e2e test run returns 0
Environment
Node : 12.16.0
OS : linux x64
npm : 6.14.8
nx : Not Found
@nrwl/angular : 11.3.1
@nrwl/cli : 11.3.1
@nrwl/cypress : 11.3.1
@nrwl/devkit : 11.3.1
@nrwl/eslint-plugin-nx : Not Found
@nrwl/express : Not Found
@nrwl/jest : 11.3.1
@nrwl/linter : 11.3.1
@nrwl/nest : Not Found
@nrwl/next : Not Found
@nrwl/node : Not Found
@nrwl/react : Not Found
@nrwl/schematics : Not Found
@nrwl/tao : 11.3.1
@nrwl/web : Not Found
@nrwl/workspace : 11.3.1
typescript : 4.0.7
Command
npm run e2e --headless
that runs cypress e2e tests doesn't return non-zero exit code on test failure. If wecompare it to protractor then it returns exit code 1 in case of e2e test failure.Current Behavior
By running
npm run e2e && echo $?
the command returns exit code 0 in case of failed e2e test.This looks like to be an issue with NX executor convertor that always maps the executor's output success flag to
true
: https://github.com/nrwl/nx/blob/11.2.x/packages/devkit/src/utils/convert-nx-executor.ts#L39-L46Expected Behavior
By running
npm run e2e --headless
on a test with failure it will return non-zero exit code. This exit code information can then be used to determine the e2e test phase result on CI, if it was success or not.Steps to Reproduce
npx create-nx-workspace --preset=angular
npm run e2e --headless && echo $?
Failure Logs
Exit code after a unsuccessful e2e test run returns 0
Environment