adamgruber / mochawesome

A Gorgeous HTML/CSS Reporter for Mocha.js
https://gitter.im/mochawesome/general
MIT License
1.06k stars 160 forks source link

mochawesome's tests get skipped when failure occurs inside runner.run callback #318

Closed gabegorelick closed 4 years ago

gabegorelick commented 4 years ago

Describe the bug This is not a bug with mochawesome. This is a bug with mochawesome's test suite. The test suite contains lots of instances of running a Mocha Runner, for example:

https://github.com/adamgruber/mochawesome/blob/34fce3104a87b5da0bb1785022a8024bcccc9447/test/reporter.test.js#L80-L84

Due to recent changes in mocha, if an assertion inside the run callback fails, the test suite will exit. Basically, mocha intercepts this uncaught exception and rethrows it, which causes Node's default uncaught exception handler to print a stack trace and exit. This is not what we want, since it means Mocha exits before running subsequent tests in the suite.

Relevant code is here: https://github.com/mochajs/mocha/blob/283b11ea1fef1344aea2a0ad01fe882efcf5e3d8/lib/runner.js#L1022-L1030

Code Reproduce Make an assertion inside a run callback fail. E.g. change failureCount.should.equal(0); to failureCount.should.equal(1);. Run the test suite and notice weird behavior.

Expected behavior Test fails with mocha output. Subsequent tests are still executed.

Screenshots

image

Environment (please complete the following information):