search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.33k stars 13.56k forks source link

nodejs_20 fails to install some native npm packages on darwin #326173

Open uPaymeiFixit opened 2 months ago

uPaymeiFixit commented 2 months ago

Describe the bug

When running a native package, for example node-rdkafka, I get a failure to build. It seems this is because nodejs_20 is built using a nix-provided gcc instead of the one shipped with Xcode.

Steps To Reproduce

Expected behavior

Package should install correctly

Screenshots

❯ nix shell 'nixpkgs#nodejs_20' -c npm install node-rdkafka
npm error code 1
npm error path /Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka
npm error command failed
npm error command sh -c node-gyp rebuild
npm error make: Entering directory '/Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka/build'
npm error   ACTION deps_librdkafka_gyp_librdkafka_target_configure deps/librdkafka/config.h
npm error checking for OS or distribution... ok^[(B (osx)
npm error checking for C compiler from CC env... failed^[(B
npm error checking for gcc (by command)... ok^[(B
npm error checking for C++ compiler from CXX env... failed^[(B
npm error checking for C++ compiler (g++)... ok^[(B
npm error checking executable ld... ok^[(B
npm error checking executable nm... ok^[(B
npm error checking executable objdump... ok^[(B
npm error checking executable strip... ok^[(B
npm error checking executable libtool... ok^[(B
npm error checking executable ranlib... ok^[(B
npm error checking for debug symbols compiler flag (-g...)... failed^[(B
npm error checking for pkgconfig (by command)... failed^[(B
npm error checking for install (by command)... failed^[(B
npm error checking for GNU ar... failed^[(B (disable)
npm error checking for PIC (by compile)... failed^[(B (disable)
npm error checking for GNU-compatible linker options... failed^[(B
npm error checking for OSX linker options... failed^[(B
npm error checking for GNU linker-script ld flag... failed^[(B
npm error checking for Solaris linker-script ld flag... failed^[(B (ignore)
npm error checking for __atomic_32 (by compile)... failed^[(B
npm error checking for __atomic_32_lib (by compile)... failed^[(B
npm error checking for __sync_32 (by compile)... failed^[(B (disable)
npm error checking for __atomic_64 (by compile)... failed^[(B
npm error checking for __atomic_64_lib (by compile)... failed^[(B
npm error checking for __sync_64 (by compile)... failed^[(B (disable)
npm error checking for socket (by compile)... failed^[(B
npm error checking for socket_nsl (by compile)... failed^[(B (fail)
npm error parsing version '0x020300ff'... ok^[(B (2.3.0)
npm error checking for librt (by compile)... failed^[(B
npm error checking for libpthread (by compile)... failed^[(B (fail)
npm error checking for c11threads (by compile)... failed^[(B (disable)
npm error checking for libdl (by compile)... failed^[(B (disable)
npm error checking for zlib (by compile)... failed^[(B (disable)
npm error checking for libcrypto (by compile)... failed^[(B (disable)
npm error checking for libsasl2 (by compile)... failed^[(B (disable)
npm error checking for libsasl (by compile)... failed^[(B (disable)
npm error checking for libzstd (by compile)... failed^[(B (disable)
npm error checking for libcurl (by compile)... failed^[(B (disable)
npm error checking for libm (by compile)... failed^[(B (disable)
npm error checking for liblz4 (by compile)... failed^[(B (disable)
npm error checking for syslog (by compile)... failed^[(B (disable)
npm error checking for rapidjson (by compile)... failed^[(B (disable)
npm error checking for crc32chw (by compile)... failed^[(B (disable)
npm error checking for regex (by compile)... failed^[(B (disable)
npm error checking for rand_r (by compile)... failed^[(B (disable)
npm error checking for strndup (by compile)... failed^[(B (disable)
npm error checking for strlcpy (by compile)... failed^[(B (disable)
npm error checking for strerror_r (by compile)... failed^[(B (disable)
npm error checking for strcasestr (by compile)... failed^[(B (disable)
npm error checking for pthread_setname_gnu (by compile)... failed^[(B (disable)
npm error checking for pthread_setname_darwin (by compile)... failed^[(B (disable)
npm error checking for pthread_setname_freebsd (by compile)... failed^[(B (disable)
npm error checking for nm (by env NM)... ok^[(B (cached)
npm error checking for getrusage (by compile)... failed^[(B (disable)
npm error
npm error
npm error
npm error make: Leaving directory '/Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka/build'
npm error gyp info it worked if it ends with ok
npm error gyp info using node-gyp@10.1.0
npm error gyp info using node@20.14.0 | darwin | arm64
npm error gyp info find Python using Python version 3.9.6 found at "/Library/Developer/CommandLineTools/usr/bin/python3"
npm error gyp info spawn /Library/Developer/CommandLineTools/usr/bin/python3
npm error gyp info spawn args [
npm error gyp info spawn args '/nix/store/pvq8wjlmm8sh7b90nx731yhid328h21g-nodejs-20.14.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm error gyp info spawn args 'binding.gyp',
npm error gyp info spawn args '-f',
npm error gyp info spawn args 'make',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka/build/config.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/nix/store/pvq8wjlmm8sh7b90nx731yhid328h21g-nodejs-20.14.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/Users/joshuagibbs/Library/Caches/node-gyp/20.14.0/include/node/common.gypi',
npm error gyp info spawn args '-Dlibrary=shared_library',
npm error gyp info spawn args '-Dvisibility=default',
npm error gyp info spawn args '-Dnode_root_dir=/Users/joshuagibbs/Library/Caches/node-gyp/20.14.0',
npm error gyp info spawn args '-Dnode_gyp_dir=/nix/store/pvq8wjlmm8sh7b90nx731yhid328h21g-nodejs-20.14.0/lib/node_modules/npm/node_modules/node-gyp',
npm error gyp info spawn args '-Dnode_lib_file=/Users/joshuagibbs/Library/Caches/node-gyp/20.14.0/<(target_arch)/node.lib',
npm error gyp info spawn args '-Dmodule_root_dir=/Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka',
npm error gyp info spawn args '-Dnode_engine=v8',
npm error gyp info spawn args '--depth=.',
npm error gyp info spawn args '--no-parallel',
npm error gyp info spawn args '--generator-output',
npm error gyp info spawn args 'build',
npm error gyp info spawn args '-Goutput_dir=.'
npm error gyp info spawn args ]
npm error gyp info spawn make
npm error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm error ###########################################################^[(B
npm error ###                  Configure failed                   ###^[(B
npm error ###########################################################^[(B
npm error ### Accumulated failures:                               ###^[(B
npm error ###########################################################^[(B
npm error  socket_nsl () ^[(B
npm error     module: socket
npm error     action: fail
npm error     reason:
npm error compile check failed:
npm error CC: CC
npm error flags: 
npm error gcc  -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -Wall -Werror _mkltmprlJ4x6.c -o _mkltmprlJ4x6.c.o -lsocket -lnsl -L/Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka/build/deps  :
npm error _mkltmprlJ4x6.c:2:10: fatal error: 'sys/types.h' file not found
npm error #include <sys/types.h>
npm error          ^~~~~~~~~~~~~
npm error 1 error generated.
npm error source: 
npm error #include <sys/types.h>
npm error #include <sys/socket.h>
npm error #include <unistd.h>
npm error void foo (void) {
npm error    int s = socket(0, 0, 0);
npm error    close(s);
npm error }
npm error
npm error  libpthread () ^[(B
npm error     module: self
npm error     action: fail
npm error     reason:
npm error compile check failed:
npm error CC: CC
npm error flags: -lpthread
npm error gcc  -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -Wall -Werror _mkltmpMmbiiT.c -o _mkltmpMmbiiT.c.o  -L/Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka/build/deps -lpthread :
npm error _mkltmpMmbiiT.c:1:10: fatal error: 'pthread.h' file not found
npm error #include <pthread.h>
npm error          ^~~~~~~~~~~
npm error 1 error generated.
npm error source: #include <pthread.h>
npm error
npm error make: *** No rule to make target '../deps/librdkafka/config.h', needed by '11a9e3388a67e1ca5c31c1d8da49cb6d2714eb41.intermediate'.  Stop.
npm error gyp ERR! build error 
npm error gyp ERR! stack Error: `make` failed with exit code: 2
npm error gyp ERR! stack at ChildProcess.<anonymous> (/nix/store/pvq8wjlmm8sh7b90nx731yhid328h21g-nodejs-20.14.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:209:23)
npm error gyp ERR! System Darwin 23.5.0
npm error gyp ERR! command "/nix/store/pvq8wjlmm8sh7b90nx731yhid328h21g-nodejs-20.14.0/bin/node" "/nix/store/pvq8wjlmm8sh7b90nx731yhid328h21g-nodejs-20.14.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm error gyp ERR! cwd /Users/joshuagibbs/Desktop/node-dyld-issue-debugging/node_modules/node-rdkafka
npm error gyp ERR! node -v v20.14.0
npm error gyp ERR! node-gyp -v v10.1.0
npm error gyp ERR! not ok

npm error A complete log of this run can be found in: /Users/joshuagibbs/.npm/_logs/2024-07-10T19_18_10_238Z-debug-0.log

Additional context

If I use a version of node from outside of nix (for example, nvm), it works correctly.

Notify maintainers

@cillianderoiste @cko

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.5.0, macOS 14.5`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.18.2`
 - channels(root): `"nixpkgs"`
 - channels(joshuagibbs): `"darwin"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

Add a :+1: reaction to issues you find important.

khaneliman commented 2 months ago

Hmm..

 nix shell 'nixpkgs#nodejs_20' -c npm install node-rdkafka

added 4 packages in 53s
npm notice
npm notice New minor version of npm available! 10.5.0 -> 10.8.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
npm notice Run npm install -g npm@10.8.2 to update!
npm notice
uPaymeiFixit commented 1 month ago

@khaneliman hmm indeed... Did you run this from an M1 or M2 mac?

khaneliman commented 1 month ago

@khaneliman hmm indeed... Did you run this from an M1 or M2 mac?

M1

uPaymeiFixit commented 1 month ago

That's interesting. I have a friend who is also able to reproduce it. Can you try running node -p require("node-rdkafka") within the nix shell? I was able to get node-rdkafka installed by installing gcc and llvm via nix, but when I required it I ran into other issues.

khaneliman commented 1 month ago

Just got the error today, I think it's related to xcode dev tools and not sandboxed on darwin. I would assume we just need to properly provide those binaries in the derivation...

image