How to properly handle forked scripts

[mwn2017]

Hello, our application forks on startup a javascript node module (let's call it "script.js") using nodejs "child_process.fork(script, args)" method. "script.js" uses addition module "diskusage".

We are not able to embed the forked script into main application executable. At present we are deployng application.exe with "script.js" (and relative modules needed by script.js in "node_modules" folder) This works well but it is not optimal.

The command "pkg ." gives this output:

pkg@4.2.4 Warning Cannot include addon %1 into executable. The addon must be distributed with executable as %2. C:\wamp64\www\playout_service\node_modules\diskusage\build\Release\diskusage.node path-to-executable/diskusage.node

Adding "script.js" into "pkg.scripts" section of package.json did not helped.

We would like to know how to properly handle this issue.

Thanks

[ronalb]

I am facing the same issue

[akobler]

Hi

I just faced the same situation. I package a repo where we use child_process.fork("childMain.js") to spin scripts in a new child process. I tried several things. My preferred solution:

If the packaged version of your node application is run, pkg installs a special directory "/snapshot", where your node process can access all source files (see: https://github.com/zeit/pkg#snapshot-filesystem). Thus, the following adaption of the fork code works for us:

var scriptPath = __dirname + "/childMain.js"
if(process.pkg) {
    scriptPath = "/snapshot/childMain.js"
}
child_process.fork(scriptPath)

Ensure that childMain.js is packaged by pkg. If not required by the main script, you must add it explicitly via package.json (see: [https://github.com/zeit/pkg#config]):(https://github.com/zeit/pkg#config)

{
  "pkg": "childMain.js"
}

Best, Andi

[ronalb]

thx @akobler , thats exactly what I intended to try to do as a workaround. I was trying to package my whole project as single executable file (or maybe 2 exe) , but fork can only take js files

Regards, Ron.

[barry-johnson]

I have found including the script in the executable works fine via explicit packaging in assets (it did not seem to get picked up from the "./lib/*/.js" scripts). However, it fails to find/load the needed native .node' file (which has always loaded fine from the main application). I have tried placing the .node file in a couple of places, both in a sparse node_modules directory structure (this is for sqlite3 so I have it at ./node_modules/sqlite3/lib/binding/node-v48-linux-arm/node_sqlite3.node and in the directory of the executable itself (i.e. ./node_sqlite3.node).

At runtime I still get an error: Error: Cannot find module '/snapshot/appname/node_modules/sqlite3/lib/binding/node-v48-linux-arm/node_sqlite3.node'.

I was thinking maybe this is because sqlite itself call require for its bindings file based on __dirname...

var binding_path = binary.find(path.resolve(path.join(__dirname,'../package.json')));
var binding = require(binding_path);

...but of course it finds the node file just fine in the main app, so I'm not sure why it appears to get hung up in the forked process. (As a detail: the file being forked doesn't live at the top of the tree, but is itself down in a /lib/services directory. Not sure if that makes a difference and in fact the main file in this case also lives a couple of levels below the project's root directory.)

In any case, I wonder if anyone has found a solution to the specific question asked by the OP: i.e. dealing with the native .node files, not just being able to spin up the forked .js file.

@akobler - It isn't clear whether your forked file (or one it references) requires any modules with .node native extensions. Does it? If so, any details you could provide about where the .node native extension file is living at runtime?

@mwn2017 - Did this approach, or another, end up working for your use case with diskusage?

[reigelgallarde]

you can also use path and it will work for you...

const { fork }                  = require('child_process');
const path                      = require('path');
forked = fork(path.join(__dirname, 'child.js'));

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[barry-johnson]

This might be considered stale, but the general question of dealing with node modules with native libraries remains unaddressed and is raised in several open issues which are getting marked as stale. Issue still seems unresolved (or if resolved, not clear from doc or notes).

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[barry-johnson]

This might be considered stale, but the general question of dealing with node modules with native libraries remains unaddressed and is raised in several open issues which are getting marked as stale. Issue still seems unresolved (or if resolved, not clear from doc or notes).

Maybe instead of marking stale, just mark it as WONTFIX if that is the decision of the maintainers.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[barry-johnson]

Again, I will repeat - this still isn't addressed. Mark it WONTFIX or give some sort of an update.