nodejs / node-gyp

Node.js native addon build tool
MIT License
9.89k stars 1.79k forks source link

AIX Native modules build #2256

Open mauroballestra opened 3 years ago

mauroballestra commented 3 years ago
Verbose output (from npm or node-gyp): ``` $ npm install ref-napi --loglevel verbose npm info it worked if it ends with ok npm verb cli [ npm verb cli '/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/node', npm verb cli '/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/npm', npm verb cli 'install', npm verb cli 'ref-napi', npm verb cli '--loglevel', npm verb cli 'verbose' npm verb cli ] npm info using npm@6.14.8 npm info using node@v14.15.0 npm verb npm-session 2f61aed209eead82 npm http fetch GET 304 https://registry.npmjs.org/ref-napi 1158ms (from cache) npm timing stage:loadCurrentTree Completed in 1402ms npm timing stage:loadIdealTree:cloneCurrentTree Completed in 1ms npm timing stage:loadIdealTree:loadShrinkwrap Completed in 1ms npm http fetch GET 304 https://registry.npmjs.org/node-gyp-build 207ms (from cache) npm http fetch GET 304 https://registry.npmjs.org/node-addon-api 279ms (from cache) npm http fetch GET 304 https://registry.npmjs.org/debug 347ms (from cache) npm http fetch GET 304 https://registry.npmjs.org/get-symbol-from-current-process-h 1029ms (from cache) npm http fetch GET 304 https://registry.npmjs.org/ms 98ms (from cache) npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 1187ms npm timing stage:loadIdealTree Completed in 1205ms npm timing stage:generateActionsToTake Completed in 19ms npm verb correctMkdir /home/fctech/.npm/_locks correctMkdir not in flight; initializing npm verb lock using /home/fctech/.npm/_locks/staging-79bd5c4ad4d29246.lock for /home/fctech/Mauro/test/node_modules/.staging npm timing audit submit Completed in 319ms npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/audits/quick 319ms npm timing audit body Completed in 3ms npm timing action:extract Completed in 1089ms npm timing action:finalize Completed in 49ms npm timing action:refresh-package-json Completed in 163ms npm info lifecycle ms@2.1.2~preinstall: ms@2.1.2 npm info lifecycle debug@4.2.0~preinstall: debug@4.2.0 npm info lifecycle get-symbol-from-current-process-h@1.0.2~preinstall: get-symbol-from-current-process-h@1.0.2 npm info lifecycle node-addon-api@2.0.2~preinstall: node-addon-api@2.0.2 npm info lifecycle node-gyp-build@4.2.3~preinstall: node-gyp-build@4.2.3 npm info lifecycle ref-napi@3.0.1~preinstall: ref-napi@3.0.1 npm timing action:preinstall Completed in 8ms npm info linkStuff ms@2.1.2 npm info linkStuff debug@4.2.0 npm info linkStuff get-symbol-from-current-process-h@1.0.2 npm info linkStuff node-addon-api@2.0.2 npm info linkStuff node-gyp-build@4.2.3 npm verb linkBins [ npm verb linkBins { npm verb linkBins 'node-gyp-build': 'bin.js', npm verb linkBins 'node-gyp-build-optional': 'optional.js', npm verb linkBins 'node-gyp-build-test': 'build-test.js' npm verb linkBins }, npm verb linkBins '/home/fctech/Mauro/test/node_modules/.bin', npm verb linkBins false npm verb linkBins ] npm info linkStuff ref-napi@3.0.1 npm timing action:build Completed in 49ms npm info lifecycle ms@2.1.2~install: ms@2.1.2 npm info lifecycle debug@4.2.0~install: debug@4.2.0 npm info lifecycle get-symbol-from-current-process-h@1.0.2~install: get-symbol-from-current-process-h@1.0.2 npm info lifecycle node-addon-api@2.0.2~install: node-addon-api@2.0.2 npm info lifecycle node-gyp-build@4.2.3~install: node-gyp-build@4.2.3 npm info lifecycle ref-napi@3.0.1~install: ref-napi@3.0.1 > ref-napi@3.0.1 install /home/fctech/Mauro/test/node_modules/ref-napi > node-gyp-build /home/fctech/Mauro/test/node_modules/node-gyp-build/index.js:59 throw new Error('No native build was found for ' + target + '\n loaded from: ' + dir + '\n') ^ Error: No native build was found for platform=aix arch=ppc64 runtime=node abi=83 uv=1 libc=glibc node=14.15.0 loaded from: /home/fctech/Mauro/test/node_modules/ref-napi at Function.load.path (/home/fctech/Mauro/test/node_modules/node-gyp-build/index.js:59:9) at load (/home/fctech/Mauro/test/node_modules/node-gyp-build/index.js:21:30) at Object. (/home/fctech/Mauro/test/node_modules/node-gyp-build/build-test.js:19:19) at Module._compile (internal/modules/cjs/loader.js:1063:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10) at Module.load (internal/modules/cjs/loader.js:928:32) at Function.Module._load (internal/modules/cjs/loader.js:769:14) at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12) at internal/main/run_main_module.js:17:47 gyp info it worked if it ends with ok gyp verb cli [ gyp verb cli '/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/node', gyp verb cli '/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js', gyp verb cli 'rebuild' gyp verb cli ] gyp info using node-gyp@5.1.0 gyp info using node@14.15.0 | aix | ppc64 gyp verb command rebuild [] gyp verb command clean [] gyp verb clean removing "build" directory gyp verb command configure [] gyp verb find Python Python is not set from command line or npm configuration gyp verb find Python Python is not set from environment variable PYTHON gyp verb find Python checking if "python" can be used gyp verb find Python - executing "python" to get executable path gyp verb find Python - executable path is "/opt/freeware/bin/python" gyp verb find Python - executing "/opt/freeware/bin/python" to get version gyp verb find Python - version is "2.7.18" gyp info find Python using Python version 2.7.18 found at "/opt/freeware/bin/python" gyp verb get node dir no --target version specified, falling back to host node version: 14.15.0 gyp verb command install [ '14.15.0' ] gyp verb install input version string "14.15.0" gyp verb install installing version: 14.15.0 gyp verb install --ensure was passed, so won't reinstall if already installed gyp verb install version is already installed, need to check "installVersion" gyp verb got "installVersion" 9 gyp verb needs "installVersion" 9 gyp verb install version is good gyp verb get node dir target node version installed: 14.15.0 gyp verb build dir attempting to create "build" dir: /home/fctech/Mauro/test/node_modules/ref-napi/build gyp verb build dir "build" dir needed to be created? /home/fctech/Mauro/test/node_modules/ref-napi/build gyp verb build/config.gypi creating config file gyp verb build/config.gypi writing out config file: /home/fctech/Mauro/test/node_modules/ref-napi/build/config.gypi gyp verb config.gypi checking for gypi file: /home/fctech/Mauro/test/node_modules/ref-napi/config.gypi gyp verb common.gypi checking for gypi file: /home/fctech/Mauro/test/node_modules/ref-napi/common.gypi gyp verb gyp gyp format was not specified; forcing "make" gyp verb node-gyp root npm_parent_directory is node_modules gyp verb node-gyp root Finding node root directory gyp verb node-gyp root in install directory, root = /home/fctech/Mauro/node-v14.15.0-aix-ppc64 gyp verb find exports file Found exports file: /home/fctech/Mauro/node-v14.15.0-aix-ppc64/include/node/node.exp gyp info spawn /opt/freeware/bin/python gyp info spawn args [ gyp info spawn args '/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/home/fctech/Mauro/test/node_modules/ref-napi/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/home/fctech/.cache/node-gyp/14.15.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/home/fctech/.cache/node-gyp/14.15.0', gyp info spawn args '-Dnode_exp_file=/home/fctech/Mauro/node-v14.15.0-aix-ppc64/include/node/node.exp', gyp info spawn args '-Dnode_gyp_dir=/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/home/fctech/.cache/node-gyp/14.15.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/home/fctech/Mauro/test/node_modules/ref-napi', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp verb command build [] gyp verb build type Release gyp verb architecture ppc64 gyp verb node dev dir /home/fctech/.cache/node-gyp/14.15.0 gyp verb `which` succeeded for `gmake` /opt/freeware/bin/gmake gyp info spawn gmake gyp info spawn args [ 'V=1', 'BUILDTYPE=Release', '-C', 'build' ] gmake: Entering directory '/home/fctech/Mauro/test/node_modules/ref-napi/build' cc '-DNODE_GYP_MODULE_NAME=nothing' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' -I/home/fctech/.cache/node-gyp/14.15.0/include/node -I/home/fctech/.cache/node-gyp/14.15.0/src -I/home/fctech/.cache/node-gyp/14.15.0/deps/openssl/config -I/home/fctech/.cache/node-gyp/14.15.0/deps/openssl/openssl/include -I/home/fctech/.cache/node-gyp/14.15.0/deps/uv/include -I/home/fctech/.cache/node-gyp/14.15.0/deps/zlib -I/home/fctech/.cache/node-gyp/14.15.0/deps/v8/include -pthread -Wall -Wextra -Wno-unused-parameter -maix64 -Wno-attributes -O3 -fno-omit-frame-pointer -MMD -MF ./Release/.deps/Release/obj.target/nothing/../node-addon-api/src/nothing.o.d.raw -c -o Release/obj.target/nothing/../node-addon-api/src/nothing.o ../../node-addon-api/src/nothing.c rm -f Release/obj.target/../node-addon-api/src/nothing.a && ar -X32_64 crs Release/obj.target/../node-addon-api/src/nothing.a Release/obj.target/nothing/../node-addon-api/src/nothing.o rm -rf "Release/nothing.a" && cp -pPRf "Release/obj.target/../node-addon-api/src/nothing.a" "Release/nothing.a" g++ '-DNODE_GYP_MODULE_NAME=binding' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DBUILDING_NODE_EXTENSION' -I/home/fctech/.cache/node-gyp/14.15.0/include/node -I/home/fctech/.cache/node-gyp/14.15.0/src -I/home/fctech/.cache/node-gyp/14.15.0/deps/openssl/config -I/home/fctech/.cache/node-gyp/14.15.0/deps/openssl/openssl/include -I/home/fctech/.cache/node-gyp/14.15.0/deps/uv/include -I/home/fctech/.cache/node-gyp/14.15.0/deps/zlib -I/home/fctech/.cache/node-gyp/14.15.0/deps/v8/include -I/home/fctech/Mauro/test/node_modules/node-addon-api -I/home/fctech/Mauro/test/node_modules/get-symbol-from-current-process-h/include -I/home/fctech/Mauro/test/node_modules/ref-napi/include -pthread -Wall -Wextra -Wno-unused-parameter -maix64 -Wno-attributes -O3 -fno-omit-frame-pointer -fno-rtti -std=gnu++1y -MMD -MF ./Release/.deps/Release/obj.target/binding/src/binding.o.d.raw -c -o Release/obj.target/binding/src/binding.o ../src/binding.cc g++ -shared -Wl,-bimport:/home/fctech/Mauro/node-v14.15.0-aix-ppc64/include/node/node.exp -pthread -Wl,-bbigtoc -maix64 -Wl,-blibpath:/usr/lib:/lib:/opt/freeware/lib/pthread/ppc64 -o Release/obj.target/binding.node Release/obj.target/binding/src/binding.o Release/obj.target/../node-addon-api/src/nothing.a collect2: fatal error: Release/obj.target/../node-addon-api/src/nothing.a: not a COFF file compilation terminated. gmake: *** [binding.target.mk:150: Release/obj.target/binding.node] Error 1 gmake: Leaving directory '/home/fctech/Mauro/test/node_modules/ref-napi/build' gyp ERR! build error gyp ERR! stack Error: `gmake` failed with exit code: 2 gyp ERR! stack at ChildProcess.onExit (/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23) gyp ERR! stack at ChildProcess.emit (events.js:315:20) gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:277:12) gyp ERR! System AIX 7.1 gyp ERR! command "/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/node" "/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" gyp ERR! cwd /home/fctech/Mauro/test/node_modules/ref-napi gyp ERR! node -v v14.15.0 gyp ERR! node-gyp -v v5.1.0 gyp ERR! not ok npm verb lifecycle ref-napi@3.0.1~install: unsafe-perm in lifecycle true npm verb lifecycle ref-napi@3.0.1~install: PATH: /home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/fctech/Mauro/test/node_modules/ref-napi/node_modules/.bin:/home/fctech/Mauro/test/node_modules/.bin:/opt/freeware/bin/:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java7_64/jre/bin:/usr/java7_64/bin:/home/fctech/bin:/usr/vacpp/bin/:/opt/freeware/bin:.:/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/ npm verb lifecycle ref-napi@3.0.1~install: CWD: /home/fctech/Mauro/test/node_modules/ref-napi npm info lifecycle ref-napi@3.0.1~install: Failed to exec install script npm timing action:install Completed in 14836ms npm verb unlock done using /home/fctech/.npm/_locks/staging-79bd5c4ad4d29246.lock for /home/fctech/Mauro/test/node_modules/.staging npm timing stage:rollbackFailedOptional Completed in 1000ms npm timing stage:runTopLevelLifecycles Completed in 19921ms npm WARN enoent ENOENT: no such file or directory, open '/home/fctech/Mauro/test/package.json' npm verb enoent This is related to npm not being able to find a file. npm verb enoent npm WARN test No description npm WARN test No repository field. npm WARN test No README data npm WARN test No license field. npm verb stack Error: ref-napi@3.0.1 install: `node-gyp-build` npm verb stack Exit status 1 npm verb stack at EventEmitter. (/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16) npm verb stack at EventEmitter.emit (events.js:315:20) npm verb stack at ChildProcess. (/home/fctech/Mauro/node-v14.15.0-aix-ppc64/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14) npm verb stack at ChildProcess.emit (events.js:315:20) npm verb stack at maybeClose (internal/child_process.js:1048:16) npm verb stack at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5) npm verb pkgid ref-napi@3.0.1 npm verb cwd /home/fctech/Mauro/test npm verb AIX 7.1 npm verb argv "/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/node" "/home/fctech/Mauro/node-v14.15.0-aix-ppc64/bin/npm" "install" "ref-napi" "--loglevel" "verbose" npm verb node v14.15.0 npm verb npm v6.14.8 npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! ref-napi@3.0.1 install: `node-gyp-build` npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the ref-napi@3.0.1 install script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm verb exit [ 1, true ] npm timing npm Completed in 22260ms npm ERR! A complete log of this run can be found in: npm ERR! /home/fctech/.npm/_logs/2020-11-09T13_39_16_050Z-debug.log ```

As suggested by the ref-napi developer (see issue 48) I wish to report the following issue I get when building the native part of ref-napi on AIX 7.1 PPC 64bit:

CC(target) Release/obj.target/nothing/../node-addon-api/src/nothing.o AR(target) Release/obj.target/../node-addon-api/src/nothing.a COPY Release/nothing.a CXX(target) Release/obj.target/binding/src/binding.o SOLINK_MODULE(target) Release/obj.target/binding.node collect2: fatal error: Release/obj.target/../node-addon-api/src/nothing.a: not a COFF file

It's relatively easy to fix the issue in the Makefile by changing the AR archive command from: cmd_alink = rm -f $@ && $(AR.$(TOOLSET)) -X32_64 crs $@ $(filter %.o,$^) to cmd_alink = rm -f $@ && $(AR.$(TOOLSET)) -X32_64 crs $@ $(filter %.o,$^) --target=aixcoff-rs6000

Unfortunately as the Makefile is created by node-gyp and I have zero experience with it I'm unable to figure out how I can change node-gyp or any other of its related files to obtain the Makefile as I need.

Can someone help by changing the way the Makefile is generated for AIX PPC?

Thanks

richardlau commented 3 years ago

The relevant generator is https://github.com/nodejs/node-gyp/blob/ee6a837cb71f465c0e689a81b4adc784bdf8a0c7/gyp/pylib/gyp/generator/make.py#L223-L238

Maybe as a workaround you could try setting AR_target to something like $(AR) --target=aixcoff-rs6000? https://github.com/nodejs/node-gyp/blob/ee6a837cb71f465c0e689a81b4adc784bdf8a0c7/gyp/pylib/gyp/generator/make.py#L2291

cc @nodejs/platform-aix

ThePrez commented 3 years ago

FYI, I tested on IBM i 7.3 (variant of AIX 7.1 TL4) with node 14.11.0 and the build was successful. We need to proceed with caution because I don't believe the IBM i ar command supports --target. We will need to do testing/verification.

mauroballestra commented 3 years ago

@ThePrez,

thank you for checking. In fact I've tried with the IBM XLC toolchain but with that I'm unable to reach the AR phase. Output: gmake: Entering directory '/home/fctech/Mauro/test/node_modules/ref-napi/build' CC(target) Release/obj.target/nothing/../node-addon-api/src/nothing.o cc: 1501-289 (W) Option -Wall was incorrectly specified. The option will be ignored. cc: 1501-289 (W) Option -Wextra was incorrectly specified. The option will be ignored. cc: 1501-289 (W) Option -Wno-unused-parameter was incorrectly specified. The option will be ignored. cc: 1501-289 (W) Option -Wno-attributes was incorrectly specified. The option will be ignored. cc: 1501-210 (W) command option t contains an incorrect subargument cc: 1501-216 (W) command option -64 is not recognized - passed to ld cc: 1501-208 (S) command option D is missing a subargument gmake: *** [../node-addon-api/src/nothing.target.mk:108: Release/obj.target/nothing/../node-addon-api/src/nothing.o] Error 40 gmake: Leaving directory '/home/fctech/Mauro/test/node_modules/ref-napi/build' Anyways you are correct as IBM ar doesn't support --target switch. May I ask you which version of IBM XLC did you used to build?

mauroballestra commented 3 years ago

@ThePrez,

thanks to your suggestion I've been able to setup an environment where I use gcc as compiler and IBM ar as archiver. In this case the build is successful for me but I'm still unsure which toolchain I am expected to use to build with node-gyp on AIX.

ThePrez commented 3 years ago

@ThePrez,

thanks to your suggestion I've been able to setup an environment where I use gcc as compiler and IBM ar as archiver. In this case the build is successful for me but I'm still unsure which toolchain I am expected to use to build with node-gyp on AIX.

I can't give an official answer, but given that the supported platforms page for Node calls out GCC 6 as what's used for the platform builds, I would guess GCC is the expected norm, at minimum.

ThePrez commented 3 years ago

May I ask you which version of IBM XLC did you used to build?

I used gcc 6.3.0

aixtools commented 3 years ago

FYI, I tested on IBM i 7.3 (variant of AIX 7.1 TL4) with node 14.11.0 and the build was successful. We need to proceed with caution because I don't believe the IBM i ar command supports --target. We will need to do testing/verification.

Just thought I'd mention that the native ar on AIX does not support --target either. Feels like coreutils are added, and in the PATH.

I've never tried to do this build - have some other priorities this week - but I'll see about what I can do independently of the current test machines and try and figure out why AR should have an affect on anything.

Have you tried not using the X32_64 argument. By default AIX is in 32-bit mode, and to swap to 64-bit mode just use: # export OBJECT_MODE=64

And all the supporting tools know that mode -X64 is in effect. (and when unset -X32 is effective the option set).