OS [e.g. Mac, Arch, Windows 10]: mac local, docker alpine, docker node:20
Node Version [e.g. 8.2.1]: 20, 22
NPM Version [e.g. 5.4.2]: 10.7.0
C++ Toolchain [e.g. Visual Studio, llvm, g++]: g++
node-rdkafka version [e.g. 2.3.3]: 2.18.0, 3.0.x
Steps to Reproduce
node-rdkafka is used internally by a custom library, which is built and npm published to our repo. (I've tried these steps with a couple different versions of node-rdkafka
This custom library is then included as a dependency of an application project. When the application project runs npm install, and it includes the dependency, it runs node-gyp, but the cflags setting doesn't correctly get set (c++-17) so it bombs out with lots of errors.
Here's the sequence of steps that trigger the issue...
nvm use 20
(cd library-dependency)
npm install (works)
publish library, all good.
(also, going to node-modules/node-rdkafka and manually running node-gyp rebuild works.
(cd application-project)
nvm use 20
npm install (fails)
npm error In file included from /root/.cache/node-gyp/20.15.1/include/node/v8-object.h:9,
npm error from /root/.cache/node-gyp/20.15.1/include/node/v8-array-buffer.h:13,
npm error from /root/.cache/node-gyp/20.15.1/include/node/v8.h:24,
npm error from /root/.cache/node-gyp/20.15.1/include/node/node.h:73,
npm error from ../../nan/nan.h:58,
npm error from /build/node_modules/node-rdkafka/src/binding.h:13,
npm error from ../src/binding.cc:11:
npm error /root/.cache/node-gyp/20.15.1/include/node/v8-maybe.h:106:45: error: 'is_lvalue_reference_v' is not a member of 'std'; did you mean 'is_lvalue_reference'?
npm error 106 | template <class U, std::enable_if_t<!std::is_lvalue_reference_v<U>>*>
npm error | ^~~~~~~~~~~~~~~~~~~~~
npm error | is_lvalue_reference
npm error In file included from ../../nan/nan.h:58:
npm error Library/Caches/node-gyp/20.15.1/include/node/node.h:692:8: error: no template named 'optional' in namespace 'std'
npm error std::optional<std::string> builder_script_path;
nvm use 22
npm install (fails, same as above)
nvm use 18
npm install (works!)
nvm use 20
add
"overrides": {
"node-rdkafka": "3.x.x"
},
to application package.json...
npm install (works!)
Additional context
The application doesn't touch node-rdkafka directly, it uses the implementation from the included library only. But adding the dependency as an override in the parent app makes everything start to work.
This is code that has worked for years without being touched, all I was trying to do was use a more modern version of node. I'd nuked package-lock and node_modules, so this was a clean install. It seems really weird to me that the "node-gyp a dependency" works in node 18 but not on node 20-22 with the exact same code. None of our other libraries that have node-gyp built code require this kind of override in the application.
Environment Information
Steps to Reproduce
node-rdkafkais used internally by a custom library, which is built and npm published to our repo. (I've tried these steps with a couple different versions ofnode-rdkafkaThis custom library is then included as a dependency of an application project. When the application project runs
npm install, and it includes the dependency, it runsnode-gyp, but the cflags setting doesn't correctly get set (c++-17) so it bombs out with lots of errors.Here's the sequence of steps that trigger the issue...
nvm use 20 (cd library-dependency) npm install (works) publish library, all good. (also, going to
node-modules/node-rdkafkaand manually runningnode-gyp rebuildworks.(cd application-project) nvm use 20 npm install (fails)
nvm use 22 npm install (fails, same as above)
nvm use 18 npm install (works!)
nvm use 20 add
to application package.json... npm install (works!)
Additional context
The application doesn't touch
node-rdkafkadirectly, it uses the implementation from the included library only. But adding the dependency as an override in the parent app makes everything start to work.This is code that has worked for years without being touched, all I was trying to do was use a more modern version of node. I'd nuked package-lock and node_modules, so this was a clean install. It seems really weird to me that the "node-gyp a dependency" works in node 18 but not on node 20-22 with the exact same code. None of our other libraries that have node-gyp built code require this kind of override in the application.