Uncaught exceptions in between Mocha tests are not reported

[robertknight]

We encountered an issue in our application where Karma would sometimes start running tests and then exit part-way through but still report that all tests had been executed successfully.

Debugging the interactions between karma-mocha and mocha, it seems that what was happening is:

1. Mocha runs a test
2. The test starts a timer or some other async task
3. The test returns before the timer has expired, Mocha reports that the test passed and karma-mocha reports this back to Karma
4. At some point after the test has "finished", the timer expires in between tests and the timer callback triggers an uncaught exception. Mocha catches the uncaught exception via an error global event handler and reports a fail event to karma-mocha, followed by an end event.
5. Since this error occurs outside of a test there is no corresponding test end event. karma-mocha only reports the test status to Karma in response to the test end event and not the fail event. As a result, the Karma test run finishes reporting that all tests executed successfully, despite having only run some of the tests

I don't yet have a complete example project, but I was able to reproduce the problem reliably in our test suite by adding this broken test:

describe('TESTING', () => {
  it('should blow up', () => {
    setTimeout(() => {
      throw new Error('Fail!');
    }, 0);

    // Test exits here and "passes", but triggers an error later in between tests
  });
});

This was tested with karma-mocha v2.0.1 and mocha v8.1.1.

The expected behavior is that the uncaught exception is reported and the test run should fail.

I'm not sure here whether the fault is really in mocha for not generating an extra test end event or whether karma-mocha should anticipate receiving a fail event that is not followed by a test end event. It would seem wise to me that karma-mocha should have logic to ensure that fail events always get reported even if mocha doesn't generate a test end event though.

[robertknight]

As a workaround we added event handlers for uncaught exceptions/promise rejections in our test suite which saves the error and re-throws it later in a context that does get reported to Karma: https://github.com/hypothesis/client/blob/915be0b927edcb9b0215fa94f8b9e61c934c3fc9/src/sidebar/test/bootstrap.js#L29

[robertknight]

A similar issue occurs with unhandled promise rejections which trigger an unhandledrejection event to be dispatched to the window. Unlike uncaught exceptions it seems that Mocha doesn't currently intercept the unhandledrejection event at all, so no fail event is reported.