hbenl / vscode-mocha-test-adapter

Mocha Test Adapter for the VS Code Test Explorer
MIT License
91 stars 31 forks source link

Parallel tests don't run (TypeScript) #205

Closed petterm-wwt closed 2 years ago

petterm-wwt commented 2 years ago

Parallel tests aren't running in the test explorer, for "all" or for any individual tests.

They run fine in parallel on the command line, AND they run fine serially in this test adapter when NOT using parallel mode.

But when I add "mochaExplorer.parallel": true to my settings, no tests will run. The output in the Test Explorer panel shows no error, just that no tests actually ran:

[2022-04-14 16:06:32.106] [INFO] Worker: Converting tests and suites
[2022-04-14 16:06:32.125] [INFO] Received tests from worker
[2022-04-14 16:06:32.258] [INFO] Worker finished with code 0 and signal null
[2022-04-14 16:06:35.257] [INFO] Running test(s) ["<testfile>"] of <path>
[2022-04-14 16:06:35.259] [DEBUG] Spawning /home/<user>/.vscode/extensions/hbenl.vscode-mocha-test-adapter-2.13.3/out/worker/bundle.js with IPC options {}
[2022-04-14 16:06:35.311] [INFO] Worker: Using the mocha package at <project>/node_modules/mocha
[2022-04-14 16:06:35.369] [INFO] Worker: Trying requireOrImport('ts-node/register')
[2022-04-14 16:06:35.887] [INFO] Worker: Loading files
[2022-04-14 16:06:35.887] [INFO] Worker: Trying to use Mocha's experimental ESM module loader
[2022-04-14 16:06:37.481] [INFO] Worker: Running tests
[2022-04-14 16:06:37.755] [INFO] Received {"type":"finished"}
[2022-04-14 16:06:37.777] [INFO] Worker finished

I am using node v16.14.0 and mocha 9.2.0.

petterm-wwt commented 2 years ago

I've dug into this more and found that there is indeed a bug here.

Parallel works in the simplest case with Javascript; however, I am using Typescript and it doesn't work in parallel, while it works in serial. I believe I have tracked this down to the root cause:

In parallel mode, in Mocha.run(...), the root Mocha instance is used to load and configure the tests, and the parallel worker instances then each create their own Mocha instances, which are used to run the tests in separate processes:

const mocha = new Mocha(opts).addFile(filepath); (source)

Note that the only means provided for the mocha configuration to be shared from the root Mocha instance to the worker instances is the options object.

However, in this extension, requires are only loaded for the root Mocha instance, and not added to the options object on the root Mocha instance, and thus not inherited by the workers:

let requires = []
        for (let req of args.mochaOpts.requires) {

            if (fs.existsSync(req) || fs.existsSync(`${req}.js`)) {
                req = path.resolve(req);
            }

            if (requireOrImport) {
                if (args.logEnabled) sendMessage(`Trying requireOrImport('${req}')`);
                requires.push(await requireOrImport(req));
            } else {
                if (args.logEnabled) sendMessage(`Trying require('${req}')`);
                requires.push(require(req));
            }
        }

        const mocha = new Mocha();

        ...

        for (const req of requires) {
            if (req.mochaHooks) {
                mocha.rootHooks(req.mochaHooks);
            }
            if (req.mochaGlobalSetup) {
                mocha.globalSetup(req.mochaGlobalSetup);
            }
            if (req.mochaGlobalTeardown) {
                mocha.globalTeardown(req.mochaGlobalTeardown);
            }
        }

(source)

And this means that the parallel worker processes' Mocha instances will not have the same requires available.

Since Typescript uses ts-node/register as a require, and this does not get inherited by parallel worker processes, they are failing internally when loading the spec files with ERR_UNKNOWN_FILE_EXTENSION.

Thus, no tests run.

This extension needs to be updated to ensure that requires loaded are handed down to the parallel worker processes through the options object.

petterm-wwt commented 2 years ago

I tested hacking this line in the extension:

const mocha = new Mocha();

(source)

to:

const mocha = new Mocha({require: args.mochaOpts.requires});

and it made the tests run.

hbenl commented 2 years ago

Fixed in 2.13.4