search

overlookmotel / livepack

Serialize live running code to Javascript
MIT License
40 stars 1 forks source link

Handle serializing native modules #79

Open overlookmotel opened 3 years ago

overlookmotel commented 3 years ago

Functions from native modules (.node) currently cannot be serialized. Livepack throws an error Failed to serialize function because it is not instrumented.

This prevents Livepack from packing apps relying on packages that use native modules e.g. bcrypt.

Supporting this would require:

  1. Hooking require() to identify when a native module has been imported, catalog any functions in the import, and record those functions (similar to how built-in modules are cataloged).
  2. When an object is encountered which was cataloged earlier, require() it, rather than trying to serialize.
  3. Include the .node files in the build output.

If the native module is stateful, the above will not be sufficient. Cannot replay all actions upon the module to get it in same state, as some calls might have side effects. Therefore would require a custom serializer for each such module.

Open question: Can native modules import from other files / dynamically-linked libs? If so, just including the native module file itself in build would not be enough.

overlookmotel commented 3 years ago

Should similarly handle WASM.

overlookmotel commented 3 years ago

ncc docs suggest that native modules can use shared libraries. See here.

ncc's mechanism for including them is to search for all possible shared libs in the package the .node file originates from and placing them in same positions relative to the .node file in output. See here.

overlookmotel commented 3 years ago

Same logic as in ncc is included in more re-usable form in @vercel/nft.