search

denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
96.65k stars 5.34k forks source link

Feature Request: loading ffi shared libs from Uint8Array directly #15700

Open andykais opened 2 years ago

andykais commented 2 years ago

The idea here is pretty simple, in addition to Deno.dlopen accepting a string or URL, lets also let it accept a Uint8Array. The reason for this is so we can start embedding shared libraries in javascript files, so that they can be downloaded at import time. This idea is already widely used with wasm binaries, so in my opinion it isnt a large stretch for ffi libraries either

[edit] Further background on this idea is that currently we do not have a great way to package ffi shared libs alongside the rest of a deno library. Node solves this with node-gyp, and I believe precompiled binaries in some cases. In this case, we can just piggy-back off of existing javascript module imports.

eliassjogreen commented 2 years ago

Loading of dynamic libraries from memory is not natively supported, at least not on unix and is quite hard to implement as there are no pre-made solutions for rust that I know of. The solution would be to transport the dll as a javascript file, much like is done with wasm and then to write it to a file manually and load. Relevant StackOverflow question about the same thing.

TLDR; not really possible

piscisaureus commented 2 years ago

On linux it could be done with memfd_create(). How to do it on OS X and Windows is an open question though.

andykais commented 2 years ago

what @eliassjogreen outlined is what I ended up solutioning. In my case, this is an experiment for sqlite, so read & write permissions are already used by default, so its not a big deal. In other ffi cases though, it would probably be more awkward to add write permissions just to write an embedded file.

I did not realize loading dynamic libs by file was done all the way down the stack, that does make this a pretty untenable idea. I can close this issue if its not likely to be solved

piscisaureus commented 2 years ago

OS X: https://stackoverflow.com/a/11822029 Windows: https://github.com/fancycode/MemoryModule

littledivy commented 1 year ago

On macOS, NSCreateObjectFileImageFromMemory is a long-depreacted API and has been a source of code loading attacks (https://hackd.net/posts/macos-reflective-code-loading-analysis/)

Apple changed it so It simply writes to disk and uses file loading. https://github.com/apple-oss-distributions/dyld/blob/3f24a36068a96722cf3acbd5087983ce658e9d70/dyld3/APIs_macOS.cpp#L154