Always rebuilding on macOS when darwin-x64 prebuild exists?

[gniezen]

Hi Mathias,

I've followed the instructions in your lovely article on Nearform to use prebuilds with N-API to create a new module for reading files from devices using MTP: https://github.com/tidepool-org/node-mtp

Now, I've set up a Travis CI build for the module successfully that creates prebuilds for both macOS and Linux, and I can see the prebuilds in my NPM. I've used the module in an Electron app, that works fine on my local machine, but doesn't build on a Circle CI machine on macOS:

/Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp: Command failed.
Exit code: 1
Command: node-gyp-build
Arguments: 
Directory: /Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@10.1.0 | darwin | x64
gyp http GET https://nodejs.org/download/release/v10.1.0/node-v10.1.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v10.1.0/node-v10.1.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v10.1.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v10.1.0/SHASUMS256.txt
gyp info spawn /usr/local/bin/python2
gyp info spawn args [ '/Users/distiller/.nvm/versions/node/v10.1.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/distiller/.nvm/versions/node/v10.1.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/distiller/.node-gyp/10.1.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/distiller/.node-gyp/10.1.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/distiller/.nvm/versions/node/v10.1.0/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/distiller/.node-gyp/10.1.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
/bin/sh: pkg-config: command not found

Now, if understood correctly from the article, if I have a darwin-x64 prebuild the CI server shouldn't have to rebuild the module for my Electron app. If this is the case, why is it still rebuilding? Please let me know what other info I can provide to troubleshoot further?

Could it be because I'm linking to a shared library? In which case, should I include the shared library in the module itself as a static library?

[mafintosh]

Hi!

Yea you need to ship the shared library with the prebuilds in this case. We do this in sodium-native, using prebuildify, https://github.com/sodium-friends/sodium-native but it's a bit cumbersome. Compiling it in as a static library should work fine as well :)

Let me know if this makes sense.

[gniezen]

Ah, that explains it, thanks!

[mafintosh]

Are you using the new n-api? Then the size overhead of statically compiling is prob negligible since you need very few builds :).

[gniezen]

I am indeed using the new N-API. The library I'm using (libmtp) is LGPL, while my Node module is BSD-2-Clause. My understanding from LGPL is that if I'm statically linking, then it is a derivative work and I may have to make it LGPL as well.

For the shared library, can I just copy over the ones installed by Homebrew, or would I need to compile my own from source, like you did with sodium-native?

[gniezen]

Just a quick follow-up: I went with just copying the dynamic library into the lib/ folder, instead of recompiling it from source every time. Then I do the following after npm run prebuild in my .travis.yml:

- if [ $TRAVIS_OS_NAME = osx ]; then
  cp "$TRAVIS_BUILD_DIR/lib/libmtp.dylib" "$TRAVIS_BUILD_DIR/prebuilds/darwin-x64/";
  install_name_tool -change "/usr/local/opt/libmtp/lib/libmtp.9.dylib" "@loader_path/libmtp.dylib" "$TRAVIS_BUILD_DIR/prebuilds/darwin-x64/node-napi.node";
  otool -L "$TRAVIS_BUILD_DIR/prebuilds/darwin-x64/node-napi.node";
  fi