search

kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
484 stars 39 forks source link

Provide prebuilds for lmdb #92

Closed aminya closed 2 years ago

aminya commented 2 years ago

Mirrored from https://github.com/parcel-bundler/parcel/issues/7146

🐛 bug report

lmdb fails to build from the source on Windows. 

node_modules/.pnpm/lmdb-store@1.6.7/node_modules/lmdb-store: Running install script, failed in 1.3s
.../node_modules/lmdb-store install$ node-gyp-build
│ C:\Users\aminy\project\node_modules\.pnpm\lmdb-store@1.6.7\node_modules\lmdb-store>if not defined npm_confi
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@8.3.0
│ gyp info using node@17.0.0 | win32 | x64
│ gyp info find Python using Python version 3.8.10 found at "C:\Python38\python.exe"
│ gyp info find VS using VS2019 (16.11.31702.278) found at:
│ gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview"
│ gyp info find VS run with --verbose for detailed information
│ gyp info spawn C:\Python38\python.exe
│ gyp info spawn args [
│ gyp info spawn args   'C:\\Users\\aminy\\AppData\\Roaming\\npm\\pnpm-global\\5\\node_modules\\.pnpm\\pnpm@6.18.0\\node_modules\
│ gyp info spawn args   'binding.gyp',
│ gyp info spawn args   '-f',
│ gyp info spawn args   'msvs',
│ gyp info spawn args   '-I',
│ gyp info spawn args   'C:\\Users\\aminy\\project\\node_modules\\.pnpm\\lmdb-store@1.6.7\\node_modules\\l
│ gyp info spawn args   '-I',
│ gyp info spawn args   'C:\\Users\\aminy\\AppData\\Roaming\\npm\\pnpm-global\\5\\node_modules\\.pnpm\\pnpm@6.18.0\\node_modules\
│ gyp info spawn args   '-I',
│ gyp info spawn args   'C:\\Users\\aminy\\AppData\\Local\\node-gyp\\Cache\\12.18.3\\include\\node\\common.gypi',
│ gyp info spawn args   '-Dlibrary=shared_library',
│ gyp info spawn args   '-Dvisibility=default',
│ gyp info spawn args   '-Dnode_root_dir=C:\\Users\\aminy\\AppData\\Local\\node-gyp\\Cache\\12.18.3',
│ gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\aminy\\AppData\\Roaming\\npm\\pnpm-global\\5\\node_modules\\.pnpm\\pnpm@6.18.0
│ gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\aminy\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\12.18.3\\\\<(target_arc
│ gyp info spawn args   '-Dmodule_root_dir=C:\\Users\\aminy\\project\\node_modules\\.pnpm\\lmdb-store@1.6.
│ gyp info spawn args   '-Dnode_engine=v8',
│ gyp info spawn args   '--depth=.',
│ gyp info spawn args   '--no-parallel',
│ gyp info spawn args   '--generator-output',
│ gyp info spawn args   'C:\\Users\\aminy\\project\\node_modules\\.pnpm\\lmdb-store@1.6.7\\node_modules\\l
│ gyp info spawn args   '-Goutput_dir=.'
│ gyp info spawn args ]
│ gyp: name 'openssl_fips' is not defined while evaluating condition 'openssl_fips != ""' in binding.gyp while trying to load bin
│ gyp ERR! configure error
│ gyp ERR! stack Error: `gyp` failed with exit code: 1
│ gyp ERR! stack     at ChildProcess.onCpExit (C:\Users\aminy\AppData\Roaming\npm\pnpm-global\5\node_modules\.pnpm\pnpm@6.18.0\no
│ gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
│ gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
│ gyp ERR! System Windows_NT 10.0.22000
│ gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\aminy\\AppData\\Roaming\\npm\\pnpm-global\\5\\node_modules\\
│ gyp ERR! cwd C:\Users\aminy\project\node_modules\.pnpm\lmdb-store@1.6.7\node_modules\lmdb-store
│ gyp ERR! node -v v17.0.0
│ gyp ERR! node-gyp -v v8.3.0
│ gyp ERR! not ok
└─ Failed in 1.3s

🎛 Configuration (.babelrc, package.json, cli command)

N/A

🤔 Expected Behavior

I should not need to build lmdb from the source.

😯 Current Behavior

lmdb is built from the source, and it fails to build.

💁 Possible Solution

Make a PR to lmdb and provide prebuilds.

🔦 Context

💻 Code Sample

🌍 Your Environment

Software Version(s)
Node 17.0.0
npm/Yarn pnpm 6.18.0
Operating System Windows 11
kriszyp commented 2 years ago

Yeah, NodeJS messed up their header includes in the v17 release: https://github.com/nodejs/Release/issues/704 They have merged the fix: https://github.com/nodejs/node/pull/40526 And I think they are fast-tracking a release to get it fixed since it breaks so many packages with native modules. I'll try to get a new lmdb/lmdb-store release out with prebuilds for v17 as soon as possible, but I thought I would wait for the 17.0.1 release fix so I could get it built across all platforms.

kriszyp commented 2 years ago

Actually, it looks like v17.0.1 is released, they just haven't updated nodejs.org yet: https://github.com/nodejs/node/commit/d68dd62bd0a0f4ce2320f017f5c63cbe5db9b916 I will try to get this built in the next couple hours.

kriszyp commented 2 years ago

Ok, hopefully v1.6.10 should have prebuilds for node v17 working now.

mistic commented 2 years ago

@kriszyp I wonder if this is also happening on node v16.11.1? I'm also experiencing the following error on a centos-7 x86_64 swarm linux metal centos with the following stack:

info [bazel] error /var/lib/jenkins/workspace/kibana/node_modules/lmdb-store: Command failed.
 info [bazel] Exit code: 1
 info [bazel] Command: node-gyp-build
 info [bazel] Arguments: 
 info [bazel] Directory: /var/lib/jenkins/workspace/kibana/node_modules/lmdb-store
 info [bazel] Output:
 info [bazel] gyp info it worked if it ends with ok
 info [bazel] gyp info using node-gyp@8.2.0
 info [bazel] gyp info using node@16.11.1 | linux | x64
 info [bazel] gyp info find Python using Python version 3.6.8 found at "/usr/bin/python3"
 info [bazel] gyp info spawn /usr/bin/python3
 info [bazel] gyp info spawn args [
 info [bazel] gyp info spawn args   '/var/lib/jenkins/.cache/bazel/_bazel_jenkins/316c7bbcea613ee57c362c5505204174/external/nodejs_linux_amd64/bin/nodejs/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
 info [bazel] gyp info spawn args   'binding.gyp',
 info [bazel] gyp info spawn args   '-f',
 info [bazel] gyp info spawn args   'make',
 info [bazel] gyp info spawn args   '-I',
 info [bazel] gyp info spawn args   '/var/lib/jenkins/workspace/kibana/node_modules/lmdb-store/build/config.gypi',
 info [bazel] gyp info spawn args   '-I',
 info [bazel] gyp info spawn args   '/var/lib/jenkins/.cache/bazel/_bazel_jenkins/316c7bbcea613ee57c362c5505204174/external/nodejs_linux_amd64/bin/nodejs/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
 info [bazel] gyp info spawn args   '-I',
 info [bazel] gyp info spawn args   '/var/lib/jenkins/.cache/node-gyp/16.11.1/include/node/common.gypi',
 info [bazel] gyp info spawn args   '-Dlibrary=shared_library',
 info [bazel] gyp info spawn args   '-Dvisibility=default',
 info [bazel] gyp info spawn args   '-Dnode_root_dir=/var/lib/jenkins/.cache/node-gyp/16.11.1',
 info [bazel] gyp info spawn args   '-Dnode_gyp_dir=/var/lib/jenkins/.cache/bazel/_bazel_jenkins/316c7bbcea613ee57c362c5505204174/external/nodejs_linux_amd64/bin/nodejs/lib/node_modules/npm/node_modules/node-gyp',
 info [bazel] gyp info spawn args   '-Dnode_lib_file=/var/lib/jenkins/.cache/node-gyp/16.11.1/<(target_arch)/node.lib',
 info [bazel] gyp info spawn args   '-Dmodule_root_dir=/var/lib/jenkins/workspace/kibana/node_modules/lmdb-store',
 info [bazel] gyp info spawn args   '-Dnode_engine=v8',
 info [bazel] gyp info spawn args   '--depth=.',
 info [bazel] gyp info spawn args   '--no-parallel',
 info [bazel] gyp info spawn args   '--generator-output',
 info [bazel] gyp info spawn args   'build',
 info [bazel] gyp info spawn args   '-Goutput_dir=.'
 info [bazel] gyp info spawn args ]
 info [bazel] gyp info spawn make
 info [bazel] gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
 info [bazel] make: Entering directory `/var/lib/jenkins/workspace/kibana/node_modules/lmdb-store/build'
 info [bazel]   CC(target) Release/obj.target/lmdb-store/dependencies/lmdb/libraries/liblmdb/midl.o
 info [bazel]   CC(target) Release/obj.target/lmdb-store/dependencies/lmdb/libraries/liblmdb/chacha8.o
 info [bazel] ../dependencies/lmdb/libraries/liblmdb/chacha8.c: In function ‘chacha8’:
 info [bazel] ../dependencies/lmdb/libraries/liblmdb/chacha8.c:65:3: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
 info [bazel]    j0  = U8TO32_LITTLE(sigma + 0);
 info [bazel]    ^
 info [bazel]   CC(target) Release/obj.target/lmdb-store/dependencies/lz4/lib/lz4.o
 info [bazel]   CXX(target) Release/obj.target/lmdb-store/src/node-lmdb.o
 info [bazel] g++: error: unrecognized command line option ‘-std=gnu++14’
 info [bazel] make: *** [Release/obj.target/lmdb-store/src/node-lmdb.o] Error 1
 info [bazel] make: Leaving directory `/var/lib/jenkins/workspace/kibana/node_modules/lmdb-store/build'
 info [bazel] gyp ERR! build error 
 info [bazel] gyp ERR! stack Error: `make` failed with exit code: 2
 info [bazel] gyp ERR! stack     at ChildProcess.onExit (/var/lib/jenkins/.cache/bazel/_bazel_jenkins/316c7bbcea613ee57c362c5505204174/external/nodejs_linux_amd64/bin/nodejs/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
 info [bazel] gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
 info [bazel] gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
 info [bazel] gyp ERR! System Linux 3.10.0-693.17.1.el7.x86_64
 info [bazel] gyp ERR! command "/var/lib/jenkins/.cache/bazel/_bazel_jenkins/316c7bbcea613ee57c362c5505204174/external/nodejs_linux_amd64/bin/nodejs/bin/node" "/var/lib/jenkins/.cache/bazel/_bazel_jenkins/316c7bbcea613ee57c362c5505204174/external/nodejs_linux_amd64/bin/nodejs/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
 info [bazel] gyp ERR! cwd /var/lib/jenkins/workspace/kibana/node_modules/lmdb-store
 info [bazel] gyp ERR! node -v v16.11.1
 info [bazel] gyp ERR! node-gyp -v v8.2.0
 info [bazel] gyp ERR! not ok
 info [bazel] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
kriszyp commented 2 years ago

@mistic Is this with the latest lmdb-store, v1.6.10? I believe I could try do some new prebuilds using a centos 7 container.

mistic commented 2 years ago

@kriszyp we are still at v1.6.8 however we can upgrade to v1.6.10 if you can do this there as we will also benefit from the darwin-arm64 builds I believe 😃

kriszyp commented 2 years ago

Ok, I published v1.6.11, and that includes prebuilds for windows x64, and linux and darwin/mac-os with x64 and arm64, where linux x64 is built on a centos 7 container (linux arm64 is built on debian 9). Let me know if you see any issues with this.

mistic commented 2 years ago

@kriszyp everything seems fine right now! Thank you for the help here!