🐛 Bug: Unhandled rejections cause parallel tests to succeed when they should fail

cinder-fish commented 3 months ago

Bug Report Checklist

[X] I have read and agree to Mocha's Code of Conduct and Contributing Guidelines
[X] I have searched for related issues and issues with the faq label, but none matched my issue.
[X] I have 'smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, my usage of Mocha, or Mocha itself.
[ ] I want to provide a PR to resolve this

Expected

mocha to run the rests and exit with a non-0 exit code

Actual

mocha exits with 0

and prints the following output

causes an unhandled rejection
    ✔ throws after 3000ms

file:///Users/Alex.Brown/projects/scratch/mocha-parallel-bug/test/unhandled.rejection.test.js:4
      throw new Error("rejected after 3000ms");
      ^

Error: rejected after 3000ms
    at Timeout._onTimeout (file:///Users/Alex.Brown/projects/scratch/mocha-parallel-bug/test/unhandled.rejection.test.js:4:13)
    at listOnTimeout (node:internal/timers:573:17)
    at process.processTimers (node:internal/timers:514:7)

Node.js v22.2.0

  long running test
    1) fails after 5000ms

which includes the failing test but not the final summary

Minimal, Reproducible Example

https://github.com/cinder-fish/mocha-parallel-bug

Essentially one test file sets up a timeout which will throw an unhandled error

And another has a failing test that delays long enough that this fires

When run with parallel: true this passes despite the failing test

When run with parallel: false this fails as expected

Versions

$ mocha --version
zsh: command not found: mocha

$ node_modules/.bin/mocha --version
10.7.0

$ node --version
v22.2.0

Additional Info

No response

JoshuaKGoldberg commented 3 months ago

See also:

2797 for an issue that this one duplicates
2640 for more details on unhandled rejections & exceptions.

...however, per comments in #2797, it looks like Mocha isn't respecting Node unhandled-rejections anymore?

$ npx mocha -n unhandled-rejections=strict; echo "mocha status: $?"                                 

  causes an unhandled rejection
    ✔ throws after 3000ms

  1 passing (84ms)

mocha status: 0

Perhaps this is a regression? Needs investigation.

JoshuaKGoldberg commented 1 month ago

Ok I played around with this more: yes, this is a bug.

When running in the default mode (without parallel), mocha status: 7 is logged: indicating Mocha failing
When running with parallel, mocha status: 0 is logged: indicating no failing exit code

The behavior between the default mode & parallel should be the same. An unhandled rejection should be logged & reported.

mochajs / mocha