Mocha runner's EventEmitter does not work correctly in parallel mode

GoldStrikeArch commented 3 years ago

Prerequisites

[ x] Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
[ x] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
[ x] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
[ x] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description

The EventEmitter does not work correctly with parallel mode. It executes callbacks after all tests done, not after event. I am using mocha programmatically like that:

runner.js

const Mocha = require("mocha");

function mochaRunParallel(file) {
  const mocha = new Mocha({
    parallel: true,
  });

  mocha.addFile(file);

  const testRunner = mocha.run(() => {
    console.log(`All Done For ${file}`);
  });

  // This event emitter will not work propertly
  testRunner.on("pass", () => {
    console.log("TEST PASSED");
  });
}

mochaRunParallel("./simpleTest.js");

simpleTest.js

const sleep = (ms) => new Promise((res) => setTimeout(res, ms));

describe("Event Emitter does not work correctly in parallel mode", () => {
  it("test 1", async () => {
    await sleep(1000);
    console.log("test 1 done");
  });

  it("test 2", async () => {
    await sleep(1000);
    console.log("test 2 done");
  });
});

Steps to Reproduce

1) Create a file "runner.js" with the code that I provided.

2) Create a file "simpleTest.js" with the code that I provided.

3) Run "node ./runner.js"

Expected behavior: [What you expect to happen]

I expected to see a console.log() after first test passed.

Actual behavior: [What actually happens]

But the actual behavior is a console.log() after all tests are done. The same behavior occurs with "failure", "test end", and other events. But if you remove "parallel" flag everything is working as expected.

Reproduces how often: [What percentage of the time does it reproduce?] It reproduces 100%.

Versions

The output of mocha --version and node node_modules/.bin/mocha --version: mocha version = 8.2.1
The output of node --version: node version = 15.6.0
Your operating system
- name and version: Windows 10
- architecture (32 or 64-bit): 64-bit
Your shell (e.g., bash, zsh, PowerShell, cmd): cmd

GoldStrikeArch commented 3 years ago

Quick update: the latest version (8.3.0) still has the same issue.

nicojs commented 3 years ago

I'm pretty sure this effect is by design. It is explained here: https://mochajs.org/#parallel-mode. Since tests are executed in parallel processes (note: processes not worker threads), they are only reported back to the main process after a test file is executed.

What is it exactly that you try to accomplish?

GoldStrikeArch commented 3 years ago

I'm pretty sure this effect is by design. It is explained here: https://mochajs.org/#parallel-mode. Since tests are executed in parallel processes (note: processes not worker threads), they are only reported back to the main process after a test file is executed.

What is it exactly that you try to accomplish?

I am trying to track the progress of tests(I am using Mocha programmatically). This is very easy to do with serial-mode, like this: runner.on ('pass', callbackFunc);

And if this behavior is not provided in parallel-mode, then there is no way to track the progress of tests?

nicojs commented 3 years ago

No there isn't at the moment. Each test runner worker process reports back only once at the end, so you'll always see them buffered.

The only workaround I can currently think of is to add a afterEach root hook and using that to report status back. to your reporter, via a socket / named pipe.

GoldStrikeArch commented 3 years ago

No there isn't at the moment. Each test runner worker process reports back only once at the end, so you'll always see them buffered.

The only workaround I can currently think of is to add a afterEach root hook and using that to report status back. to your reporter, via a socket / named pipe.

I'm not sure if I fully understand this part:"using that to report status back. to your reporter, via a socket / named pipe." In this case, we still won't be able to tell which test ended, right?

JoshuaKGoldberg commented 10 months ago

Per #5027, we're not looking to make major changes to Mocha at this point. Which would leave just the type: question parts of this thread.

we still won't be able to tell which test ended, right?

From https://mochajs.org/#available-root-hooks:

As with other hooks, this refers to to the current context object

...so you should be able to view the test information from this.

If I'm wrong about that, please do file a new issue with the docs template. We should treat me being wrong here as a docs bug. 🙂

mochajs / mocha