search

Blizzard / node-rdkafka

Node.js bindings for librdkafka
MIT License
2.1k stars 390 forks source link

Kafka Consumer + kerbos authentication +GSSAPI for node js application -> broker transport failure on Consumer end #990

Open dileep006 opened 1 year ago

dileep006 commented 1 year ago

We are trying to use Kafka Conssumer + kerbos authentication +GSSAPI for node js application for enterprise chat applictaion. We are getting below error . What could be the reason for this error , any help is highly appreciated . if you have any referennce for similar implementaion , please share ...

broker transport failure.

It is working fine for JAVA client , failing from NODE JS Client.

this is our client configurations. const kafkaConsumer = new KafkaConsumer({ 'client.id': ['myserver', hostname()].join('-'), 'group.id': 'kf__write_test', 'rebalance_cb': true, 'metadata.broker.list': [servers], 'security.protocol': "sasl_ssl", 'sasl.mechanisms': 'GSSAPI', 'sasl.password': 'sdadsad', 'ssl.ca.location': path.resolve(dirname, '../../cache/certs/...'), "sasl.kerberos.service.name": "", "sasl.kerberos.principal": ", "sasl.kerberos.keytab": path.resolve(dirname, '../../cache/certs/..'), "sasl.kerberos.kinit.cmd": "%{sasl.kerberos.keytab}", "debug": "protocol,security,broker" }, { 'auto.offset.reset': 'beginning', })

Environment Information

Steps to Reproduce

node-rdkafka Configuration Settings

this is our client configurations. const kafkaConsumer = new KafkaConsumer({ 'client.id': ['myserver', hostname()].join('-'), 'group.id': 'kf__write_test', 'rebalance_cb': true, 'metadata.broker.list': [servers], 'security.protocol': "sasl_ssl", 'sasl.mechanisms': 'GSSAPI', 'sasl.password': 'asdasda', 'ssl.ca.location': path.resolve(dirname, '../../cache/certs/...'), "sasl.kerberos.service.name": "", "sasl.kerberos.principal": ", "sasl.kerberos.keytab": path.resolve(dirname, '../../cache/certs/..'), "sasl.kerberos.kinit.cmd": "%{sasl.kerberos.keytab}", "debug": "protocol,security,broker" }, { 'auto.offset.reset': 'beginning', })

Error Message

image

Additional context

image

Error logs 2022-12-19T-debug.log

320 timing command:install Completed in 41507ms 321 verbose stack Error: command failed 321 verbose stack at ChildProcess. (/Users//.nvm/versions/node/v16.10.0/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27) 321 verbose stack at ChildProcess.emit (node:events:390:28) 321 verbose stack at maybeClose (node:internal/child_process:1064:16) 321 verbose stack at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5) 322 verbose pkgid node-rdkafka@2.14.0 323 verbose cwd /Users//APP/APP/chat-socket-server 324 verbose Darwin 21.6.0 325 verbose argv "/Users//.nvm/versions/node/v16.10.0/bin/node" "/Users//.nvm/versions/node/v16.10.0/bin/npm" "install" "debug" 326 verbose node v16.10.0 327 verbose npm v7.24.0 328 error code 1 329 error path /Users//APP/APP/chat-socket-server/node_modules/node-rdkafka 330 error command failed 331 error command sh -c node-gyp rebuild 332 error Debugger attached. 332 error gyp info it worked if it ends with ok 332 error gyp info using node-gyp@8.4.1 332 error gyp info using node@16.10.0 | darwin | x64 332 error gyp info find Python using Python version 3.10.9 found at "/Library/Frameworks/Python.framework/Versions/3.10/bin/python3" 332 error gyp info spawn /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 332 error gyp info spawn args [ 332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/gyp/gyp_main.py', 332 error gyp info spawn args 'binding.gyp', 332 error gyp info spawn args '-f', 332 error gyp info spawn args 'make', 332 error gyp info spawn args '-I', 332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/build/config.gypi', 332 error gyp info spawn args '-I', 332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/addon.gypi', 332 error gyp info spawn args '-I', 332 error gyp info spawn args '/Users//Library/Caches/node-gyp/16.10.0/include/node/common.gypi', 332 error gyp info spawn args '-Dlibrary=shared_library', 332 error gyp info spawn args '-Dvisibility=default', 332 error gyp info spawn args '-Dnode_root_dir=/Users//Library/Caches/node-gyp/16.10.0', 332 error gyp info spawn args '-Dnode_gyp_dir=/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp', 332 error gyp info spawn args '-Dnode_lib_file=/Users//Library/Caches/node-gyp/16.10.0/<(target_arch)/node.lib', 332 error gyp info spawn args '-Dmodule_root_dir=/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka', 332 error gyp info spawn args '-Dnode_engine=v8', 332 error gyp info spawn args '--depth=.', 332 error gyp info spawn args '--no-parallel', 332 error gyp info spawn args '--generator-output', 332 error gyp info spawn args 'build', 332 error gyp info spawn args '-Goutput_dir=.' 332 error gyp info spawn args ] 332 error Debugger attached. 332 error Waiting for the debugger to disconnect... 332 error gyp: Call to 'node ./util/get-env.js BUILD_LIBRDKAFKA 1' returned exit status 0 while in binding.gyp. while trying to load binding.gyp 332 error gyp ERR! configure error 332 error gyp ERR! stack Error: gyp failed with exit code: 1 332 error gyp ERR! stack at ChildProcess.onCpExit (/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/lib/configure.js:259:16) 332 error gyp ERR! stack at ChildProcess.emit (node:events:390:28) 332 error gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) 332 error gyp ERR! System Darwin 21.6.0 332 error gyp ERR! command "/Users//.nvm/versions/node/v16.10.0/bin/node" "/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/.bin/node-gyp" "rebuild" 332 error gyp ERR! cwd /Users//APP/APP/chat-socket-server/node_modules/node-rdkafka 332 error gyp ERR! node -v v16.10.0 332 error gyp ERR! node-gyp -v v8.4.1 332 error gyp ERR! not ok 332 error Waiting for the debugger to disconnect... 333 verbose exit 1

dileep006 commented 1 year ago

adding a note SASL with plain text is workign fine with node-rdkafka , SASL with GSSAPI is failing with broker transport failure

const kafkaConsumer = new KafkaConsumer({ 'client.id': ['myserver', hostname()].join('-'), 'group.id': 'kf_ts_sh_cvdshp_write_test', 'rebalance_cb': true, 'metadata.broker.list': '', 'security.protocol': "sasl_plaintext", 'sasl.mechanisms': 'PLAIN', 'sasl.password': ' 'sasl.username': '', "debug": "protocol,security,broker" }, { 'auto.offset.reset': 'beginning', })

lizylgit commented 1 year ago

Did you get this working?