Closed itsame-luigi closed 3 years ago
Wow! Not used to people dropping new FFIs unannounced heh.
I was holding off on node native until we figured out either a cdecl or possibly Neon (https://neon-bindings.com/) in order to bind directly to the binary, but hadn't considered just polyfilling the WASM. That is... not a bad idea, as it removes the binary dependency which I had no clue how we were going to bundle anyways, and works fine as a stopgap while we figure that out, especially if this "just works" for both.
If I don't get this reviewed in the next few days, feel free to poke me. Life has been stupid busy lately so I'm running behind on a lot (see also: the Java FFI PR :| ), but I'm definitely curious to check this out.
This doesn't polyfill wasm, just shims the WebSocket needed by the ffi. It's purely dependent on Node's experimental wasm module support. I originally started with just trying to load it in Node to see what did and didn't work, and kept tinkering until I had something that was working. Unfortunately, wasm-pack
and pbjs
don't quite emit correct ES Module output, which is why the modularize
script was needed.
NodeJS ES modules support top-level await
, so in theory you could load the wasm using the WebAssembly
global, but wasm-pack
doesn't generate output like this. I'm not familiar enough with wasm-pack
or wasm-bindgen
to know if its possible to generate a different output that could just do something like WebAssembly.instantiate(fs.readFileSync(...))
to load the wasm file. At the very least, using --experimental-wasm-modules
is an improvement over no support at all.
This adds NodeJS support to the ffi for JavaScript using the following techniques:
.js
file extension, which is required for ESM modules."type": "module"
to treat.js
files as modules."imports"
map to handle runtime redirection of the wasm interop bindings for NodeJS.scripts/modularize.cjs
) that makes some small changes to the outputs fromprotobufjs
andwasm-pack
to make the outputs consumable both by NodeJS and Webpack, including introducing a shim formodule.require
when running as an ES module in NodeJS.websocket
to provide a W3C compatible websocket implementation when running in NodeJS.I've tested this locally using
scripts/test_node.cjs
against NodeJS 15.14.0, but it should work in the most recent releases of NodeJS 12, 14, and 16 as well. I've also tested this in the browser using the existingindex.html
.A few important notes:
syncWebAssembly: true
, as one of the imported scripts becomes "deferred" and evaluates after the main import that is supposed to be exported as the UMD global, which overwrites the value intended for the UMD global. As a result, I would not recommend upgrading to Webpack 5.x any time soon, as it will break the project even without the changes I made to support NodeJS.modularize.cjs
script fixes an issue with the output frompbjs -w es6
, which is supposed to emit ES module output, but fails to referenceprotobufjs/minimal
using a.js
extension. This is something thatprotobufjs
should fix by either adding an"exports"
map, or amending their output to include the extension. Until that time, its unfortunately necessary to modify the result.