"Error: Module did not self-register" with Node v12 worker threads

asztal commented 5 years ago

I'm finding that node-sass is failing to load when I'm using mocha-parallel-tests. It turns out that mocha-parallel-tests uses worker threads in Node v12, and that seems to be the reason. The first thread to require("node-sass") succeeds, but the second one fails with Error: Module did not self-register.

It seems reasonable that someone might want to use node-sass in worker threads, e.g. something similar to thread-loader. In my case, I'm running integration tests (in parallel) where some SCSS compilation is part of the thing being tested.

Similar error messages have been reported before, but they precede the introduction of worker threads so I presume they are caused to something else.

Minimal reproduction:

const { Worker } = require("worker_threads");

function render(element) {
    return new Promise((resolve, reject) => {
        const worker = new Worker(`
            const sass = require("node-sass");
            const data = "${element} { font-weight: bold }";
            const css = sass.renderSync({ data }).css.toString("utf8");
            console.log(css);
        `, { eval: true });
        worker.on("error", reject);
        worker.on("exit", exitCode => {
            if (exitCode > 0)
                reject(new Error(`Worker exited with code ${exitCode}`));
            resolve();
        })
    });
}

// Spawn the workers in sequence, not in parallel, to make it easier to debug
async function bug() {
    await render("b");
    await render("strong");
}

bug().catch(err => console.error(err));

Output:

lee@lee-desktop:~/Work/restapi$ node sassbug.js 
b {
  font-weight: bold; }

internal/modules/cjs/loader.js:977
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: Module did not self-register.
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:977:18)
    at Module.load (internal/modules/cjs/loader.js:790:32)
    at Function.Module._load (internal/modules/cjs/loader.js:703:12)
    at Module.require (internal/modules/cjs/loader.js:830:19)
    at require (internal/modules/cjs/helpers.js:68:18)
    at module.exports (/home/lee/Work/restapi/node_modules/node-sass/lib/binding.js:19:10)
    at Object.<anonymous> (/home/lee/Work/restapi/node_modules/node-sass/lib/index.js:14:35)
    at Module._compile (internal/modules/cjs/loader.js:936:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
    at Module.load (internal/modules/cjs/loader.js:790:32)

Versions

NPM version (npm -v): 6.11.2
Node version (node -v): v12.10.0

Node Process (node -p process.versions):

node: '12.10.0',
v8: '7.6.303.29-node.16',
uv: '1.31.0',
zlib: '1.2.11',
brotli: '1.0.7',
ares: '1.15.0',
modules: '72',
nghttp2: '1.39.2',
napi: '4',
llhttp: '1.1.4',
http_parser: '2.8.0',
openssl: '1.1.1c',
cldr: '35.1',
icu: '64.2',
tz: '2019a',
unicode: '12.1'
}

Node Platform (node -p process.platform): linux
Node architecture (node -p process.arch): x64

node-sass version (node -p "require('node-sass').info"):

node-sass       4.12.0  (Wrapper)       [JavaScript]
libsass         3.5.4   (Sass Compiler) [C/C++]

npm node-sass versions (npm ls node-sass): └── node-sass@4.12.0
Binding: I installed the binding using standard npm install, but I also tried https://github.com/sass/node-sass/releases/download/v4.12.0/linux-x64-72_binding.node (they have the same SHA256 hash anyway)

asztal commented 5 years ago

I tried switching NODE_MODULE for NAN_MODULE_WORKER_ENABLED - it compiles, works, and only segfaults 10% of the time! OK, maybe 25%. I'm counting that as a success anyway :shipit:

lee@lee-desktop:~/Work/restapi$ node sassbug.js 
b {
  font-weight: bold; }

Segmentation fault (core dumped)

I sort of expected this to work given that node-sass doesn't have any global state that I can see. Maybe libsass is doing something not thread-safe, but I wouldn't know about that.

saper commented 5 years ago

We currently do not support workers - our add on is not context-aware.

We are using libuv lowlevel routines and it might be difficult to make it worker-aware. Can you try to produce a backtrace from your crash?

asztal commented 5 years ago

Hi, I will give this a try! I did build node-sass with debugging on and didn't get much useful, but I'll try some more when I get a chance.

It didn't seem like node-sass used much global state, so I naïvely assumed it might be okay if I Just changed to NAN_MODULE_WORKER_ENABLED - but it's been a long time since I wrote any node native modules and it's changed a lot :)

saper commented 5 years ago

We have some static initialization that is done by C++ runtime for us.

An example of a problem that was caused by static initialization was this https://github.com/sass/node-sass/issues/1283#issuecomment-169845062

However we run libsass code in a background thread (outside of the main JavaScript loop) so maybe there is something we can investigate.

sass / node-sass