Use node-gyp for build?

[twuky]

Was there ever a version of this that used node-gyp in the build pipeline?

Went to try and bundle a project for Windows using pkg - pkg seems to handle the native addon poorly on windows. There's no problem packaging and running a node-raylib app on linux for me, but on Windows you get a Error: A dynamic link library (DLL) initialization routine failed. - spotted in https://github.com/RobLoach/node-raylib/issues/74. So I went to look more into this - AFAIK raylib doesn't rely on any local dlls, so I figure it's an issue loading the .node file. I tried using pkg on a file that directly references a path to the built .node file with the same issue. By default pkg tries to resolve required .node addons and include them in the build, but you can force it not to include a file (and therefore search for the addon at runtime by relative path) by using string concatenation ie: require('path/to/raylib.node' + ''), which still produces the error.

So looking further into it, the most I can find that might apply here are https://github.com/vercel/pkg/issues/305 and https://github.com/awslabs/aws-crt-nodejs/issues/227, neither of which really offer hard solutions. The latter someone suggests that the DLL issue may be a c++ runtime missing, but it stands to reason if I have the tools to build the native addon I would also have those runtimes installed? The other seems to suggest that pkg has issues specifically with native addons built via cmake-js. Im wondering if its possible to build in roughly the same way using node-gyp as a sanity check here, to see if the addon plays better with pkg.

I have found that https://github.com/leafac/caxa does work to bundle an application with node-raylib on Windows, which for now might be the best solution. In order to use this I first had to transpile the raylib applications code with ncc to resolve all dependencies intoto a single file (minus the .node which is placed in a nearby folder). caxa has some issues in that it does not build cross platform, nor does it use v8 bytecode like pkg does to add a layer of abstraction to your apps source, which may or may not be a requirement for some production apps/games. It may be possible to do that by adding something like https://github.com/bytenode/bytenode to the build process.

[RobLoach]

I think it mostly used https://github.com/nodejs/node-addon-api because the API was simpler.

[konsumer]

It might make sense to use caxa, if it works. I have had tons of problems in various projects, with pkg & ncc (mostly around half-implemented ES6 & native module stuff.) If I remember right, ncc just defers native modules (as in you need to copy the node_modules folder in the right shape into the target dir.) Esbuild (as an alternative to ncc) is also a great option for squishing all the source files into a big bundle (use --bundle flag). It's very fast, and still has support for plugins and stuff (so you can customize how it imports files, etc.) It appears it has issues with .node native modules too, though, so it might not work great.

This is more for end-users, right? Maybe it's fine to just say "pkg doesn't have very good native module support, use caxa." We could make a note about it in README. pkg is very popular, but basically every project I have tried to use it with had issues with native modules (some really huge ones, like serial, etc.)

[twuky]

I am mostly interested to see if any difference between cmake-js and node-gyp improves the ability to load it inside a PKG executable. One user in one of the issues I read seemed to claim their native addon had this error when using cmake-js but switching to gyp fixed it.

node-addon-api in their main docs examples show configuring it with a gypfile so it should be doable, if it can also include the raylib source lib. it may be worth a shot, if it makes it unilaterally easier to bundle the addon into a portable executable.

So I may take a look this week to see if I can get a gypfile that can build node-raylib, to see if that version is able to be loaded by PKG executables.

if that doesn't end up being the case, then the best route is probably just to document what methods work. it may be helpful to have some starter template with build scripts available, but the downside of caxa is that building becomes a more platform specific task, though you may be able to set up a CI action to help there.

[konsumer]

I am fairly sure the issue isn't if it was built with gyp or cmake. As far as I understand it, they are just 2 different build systems, and produce the same thing, pretty much. pkg needs native modules to be in the right shape to work (same shape as they are in your project, so like node_modules/raylib/build/Release/node-raylib.node and it doesn't matter how it's built, it doesn't inline that.

Running this on mac:

npx pkg demo.js

This worked fine and produced 3 runtimes (windows, mac, linux) and they need node_modules/raylib/build/Release/node-raylib.node built for the current platform, and as long as you have that file included, it works fine. I had similar issues with other native libs that were built with node-gyp (serial, for example.)

The pkg README says this:

Native addons (.node files) use is supported. When pkg encounters a .node file in a require call, it will package this like an asset. In some cases (like with the bindings package), the module path is generated dynamicaly and pkg won't be able to detect it. In this case, you should add the .node file directly in the assets field in package.json.

So, you just have to include it as an "asset" (or a file that is separate from your runtime, that goes in the folder that you give to users, in the same shape it was in originally.) The warning about bindings is about dynamic filename for the native module (we don't have that) so we should be ok.

I think we could even create a packager script that lets you leverage our pre-builds to make a ready-made "distro" archive of users games, since pkg does the compile & grabs correct nodejs runtime. Basically just run pkg, download 3 .node files, and zip each into a package (along with other assets.) Not sure how cross-arch works though, like I am on arm64, and I built 3 runtimes, and the mac one ran on my computer, but maybe it wouldn't work on a x64 mac.

[twuky]

its worth reinvestigating again - but i specifically remember having issues with pkg only on windows. the pkg build worked great for me on linux. i honestly cant remember how well i had set it up, or for example using the windows-build of the native addon. but during a jam game that needed a windows build i had to move the windows release to use caxa instead

[konsumer]

I have had many problems with pkg & ncc (the code bundler they made, that used to be used in pkg) and in general it does very bad things with the kinds of projects I make (current nodejs/edge-workers, native esm, less/no building, no babel, esbuild if absolutely necessary, native modules, etc.) I don't think it's a bad tool, just maybe they happen to target a code-demographic that is not really what I tend to work with.

It seems like it could work, though (tested on mac and linux on 2 arches and it worked ok with this project, at least.) I would be interested in testing with windows (need to setup a testing vmachine.)

I see your point that it could be very important, since I think a lot of users will package their game this way.