`unix-dgram` optionality problem using specific Node.js Docker image

[pintarj]

When using Janode in Docker containers I've faced the same problem as in issue #10:

node:internal/errors:477
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'unix-dgram' imported from /app/node_modules/janode/src/transport-unix.js
    at new NodeError (node:internal/errors:387:5)
    at packageResolve (node:internal/modules/esm/resolve:951:9)
    at moduleResolve (node:internal/modules/esm/resolve:1000:20)
    at defaultResolve (node:internal/modules/esm/resolve:1214:11)
    at nextResolve (node:internal/modules/esm/loader:165:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:844:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:431:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40)
    at link (node:internal/modules/esm/module_job:75:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

The issue occurs when the Docker image is built from the base image node:16-slim. Meanwhile it works well and as expected when using the node:16 image.

Steps to reproduce

1) npm init with default values 2) npm insall janode 3) add the "type": "module" line to the package.json

The package.json file should look like this:

{
  "name": "janode-issue",
  "version": "1.0.0",
  "description": "",
  "type": "module", // added line
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "janode": "^1.6.4"
  }
}

3) Create the main.js file with the following content:

import Janode from 'janode'

4) Create the Dockerfile file with the following content:

FROM node:16-slim
WORKDIR /app/
COPY main.js ./
COPY package*.json ./
RUN npm install
CMD ["node", "main.js"]

5) Build the Docker image:

docker build --tag xyz .

6) Run the built Docker image:

docker run --rm xyz

7) Error should be occured.

Appendix

The zip with all the mentioned files (plus package-lock.json file): janode-issue.zip

[atoppi]

The unix-dgram dependency needs Python in order to compile a native module. Unfortunately the node:16-slim image does not include Python. Being unix-dgram an optional dependency, the building of janode will not fail.

If you want to exclude unix-dgram please build janode with

npm install --omit=optional

Here is the output of npm when trying to install the module

root@76041d0a22c2:/home/node/janode-issue# npm install unix-dgram
npm ERR! code 1
npm ERR! path /home/node/janode-issue/node_modules/unix-dgram
npm ERR! command failed
npm ERR! command sh /tmp/install-6d1c572f.sh
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.0.0
npm ERR! gyp info using node@16.17.1 | linux | x64
npm ERR! gyp ERR! find Python 
npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
npm ERR! gyp ERR! find Python checking if "python3" can be used
npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
npm ERR! gyp ERR! find Python checking if "python" can be used
npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
npm ERR! gyp ERR! find Python 
npm ERR! gyp ERR! find Python **********************************************************
npm ERR! gyp ERR! find Python You need to install the latest version of Python.
npm ERR! gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
npm ERR! gyp ERR! find Python you can try one of the following options:
npm ERR! gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
npm ERR! gyp ERR! find Python   (accepted by both node-gyp and npm)
npm ERR! gyp ERR! find Python - Set the environment variable PYTHON
npm ERR! gyp ERR! find Python - Set the npm configuration variable python:
npm ERR! gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
npm ERR! gyp ERR! find Python For more information consult the documentation at:
npm ERR! gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
npm ERR! gyp ERR! find Python **********************************************************
npm ERR! gyp ERR! find Python 
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: Could not find any Python installation to use
npm ERR! gyp ERR! stack     at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:330:47)
npm ERR! gyp ERR! stack     at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:159:21)
npm ERR! gyp ERR! stack     at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:202:16)
npm ERR! gyp ERR! stack     at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:294:16)
npm ERR! gyp ERR! stack     at exithandler (node:child_process:408:5)
npm ERR! gyp ERR! stack     at ChildProcess.errorhandler (node:child_process:420:5)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:289:12)
npm ERR! gyp ERR! stack     at onErrorNT (node:internal/child_process:478:16)
npm ERR! gyp ERR! stack     at processTicksAndRejections (node:internal/process/task_queues:83:21)
npm ERR! gyp ERR! System Linux 5.15.0-48-generic
npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/node/janode-issue/node_modules/unix-dgram
npm ERR! gyp ERR! node -v v16.17.1
npm ERR! gyp ERR! node-gyp -v v9.0.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-10-10T09_37_12_228Z-debug-0.log

[pintarj]

But I’m not installing/building unix-dgram at all. Isn’t there a way that janode check the presence of the optional package (can dynamic import do that?).

[atoppi]

npm is installing (at least it is trying to install) every janode dependency for you.

Currently there is no check for the optional package. Dynamic import should work (e.g. loading the module when a unix transport is being requested) but I'd prefer trying other strategies before considering it, since dynamic importing does not cope well with tools like bundlers, static analyzers and so on.

For example, when bundling in browsers, we replace transport-unix.js with a stub module. If you got any suggestion about the best strategy to adopt here, please feel free to share your ideas.

[atoppi]

I guess since we switched to es modules and every dep is now being statically imported asap, making unix-dgram an optional dependency in package.json does not make any sense. Skipping that dep with --omit=optional will just trigger a failure when starting janode and it will be also be a kind of "silent" failure.

I'll move unix-dgram to the standard deps in order to have an error message when installing janode and avoid silent failures like you experienced.

[atoppi]

done in https://github.com/meetecho/janode/commit/ee0d97b10f4c9421a58f1fbb11bf0e117a1718b1

[pintarj]

Thanks for the patch, at least now it doesn't silently fail anymore. But forces Docker images to have some tools (python, make, g++, ...) installed, adding hundreds of MB, just to install an "optional" dependency that one doesn't uses. It's not a critic, just a recap.

I'll try to think about a solution that doesn't includes the dynamic import.

[atoppi]

closing for now, I'm still open to suggestions for optional dependencies