search

nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
106.51k stars 29.02k forks source link

Node 13 on MacOS cannot dyld to /usr/local/lib #32740

Open josephg opened 4 years ago

josephg commented 4 years ago

I'm running into an issue where node 13 cannot dyld to a binary in /usr/local/lib.

Context: I'm maintaining the nodejs foundationdb bindings. The way the native module works is nodejs loads a local napi module, which in turn looks for a dynamic library (libfdb_c.dylib on macos). This is normally added by the fdb installer into /usr/local/lib.

This works fine under node 8 through to node 12, but under node 13 dyld can't find the dynamic library when it loads the native module.

Weirdly it works in my CI environment in travisci, which confuses me greatly.

I can't find the code in nodejs which manages DYLD search paths (or where / how that might have changed in node 13). I can probably manually bisect node 13 versions using nvm if need be, but I'm hoping some other people have some ideas of things to try.

And it starts working again if I manually add DYLD_LIBRARY_PATH=/usr/local/lib to my environment. But I'd really rather avoid requiring that my users do that too.

$ node
Welcome to Node.js v13.12.0.
Type ".help" for more information.
> require('foundationdb')
Could not load native module. Make sure the foundationdb client is installed and
(on windows) in your PATH. https://www.foundationdb.org/download/
Uncaught:
Error: dlopen(/Users/josephg/temp/node_modules/foundationdb/prebuilds/darwin-x64/node.napi.node, 1): Library not loaded: libfdb_c.dylib
  Referenced from: /Users/josephg/temp/node_modules/foundationdb/prebuilds/darwin-x64/node.napi.node
  Reason: image not found
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1197:18)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1036:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at load (/Users/josephg/temp/node_modules/node-gyp-build/index.js:21:10)
richardlau commented 4 years ago

For the latest releases on all supported lines (that is 13.12.0, 12.16.2 and 10.20.0) we've had to switch the macOS hosts that build the releases to macOS 10.15 with a later XCode to support notarization (https://github.com/nodejs/node/issues/29216, changes in https://github.com/nodejs/node/pull/31459). This included hardening the node binary.

Would it be possible to test with 13.11.0 (last 13.x release before we switched) and/or 12.16.2 or 10.20.0 (both of which were also hardened/notarized) and report the results?

josephg commented 4 years ago

Thats probably it -

richardlau commented 4 years ago

@josephg Thanks for testing.

@nodejs/platform-macos Looks like the recent notarization/hardening is responsible. FYI @nodejs/build

AshCripps commented 4 years ago

Look like Apple has really restricted where you can load libraries from with notorizations.

https://forums.developer.apple.com/thread/124810

https://stackoverflow.com/questions/57667467/dylib-library-not-loaded-due-to-restricted-binary-after-apple-code-signing

https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html

Beginning with macOS 10.10.4, Gatekeeper verifies that no libraries are loaded from outside an app bundle. If an app uses @rpath or an absolute path to link to a dynamic library outside of the app, Gatekeeper rejects the app.

rvagg commented 4 years ago

I can't look into this atm but if anyone wants to research more the entitlements we sign with are in https://github.com/nodejs/node/blob/master/tools/osx-entitlements.plist, which I think is pretty much everything we can get. But maybe there's more we could add?

fusion2004 commented 4 years ago

I'm running into a similar issue trying to use https://github.com/dgurkaynak/nodeshout

After upgrading to 10.20.0/10.20.1, I get:

Error: Dynamic Linking Error: dlopen(libshout.dylib, 2): image not found

Setting DYLD_LIBRARY_PATH=/usr/local/lib env variable doesn't seem to resolve it for me, though. (libshout.dylib is in /usr/local/lib, where homebrew installed a symlink for it)

blackdynamo commented 4 years ago

Is there any update on this issue? We have tried many different potential workarounds with no success. We have our own .so file that we dump into the /usr/local/lib directory and prior to the notarization changes, this worked fine on Mac. We are trying to move to latest versions of node and we are stuck because of this issue.

We have tried failed with the following:

Node versions tested:

    dlopen(/path/to/addon.node, 1): Library not loaded: shared.so
      Referenced from: /path/to/addon.node
      Reason: image not found
blackdynamo commented 4 years ago

Update: We were able to find a workaround that works on MacOS in our case, and it might help others.

We ended up using the install_name_tool tool. It allows us to change the location the addon is looking at for the shared library. An example would be:

install_name_tool -change shared.so /path/to/shared.so /path/to/addon.node

We incorporate this into the install process of our module that is required by other packages and allows it to work both inside the module and the module as a dependency. Hope this helps others as well.