tulios / kafkajs

A modern Apache Kafka client for node.js
https://kafka.js.org
MIT License
3.75k stars 527 forks source link

Unable to connect to broker using jump host #1636

Open thoefkens opened 1 year ago

thoefkens commented 1 year ago

Describe the bug I am trying to connect to an MSK AWS broker using KafkaJS - I connect to an EC2 Jump Host in order to do so with AWS SSM and perform port forwarding (ssh -L 9098:brokerurl:9098).

import { Kafka, KafkaConfig } from 'kafkajs';
const {
    Type,
    awsIamAuthenticator,
} = require('@jm18457/kafkajs-msk-iam-authentication-mechanism')
const provider = awsIamAuthenticator({
    region: 'eu-central-1'
})

const kafkaConfig: KafkaConfig = {
    sasl: {
        mechanism: Type,
        authenticationProvider: provider
    },
    ssl: {
        checkServerIdentity: () => undefined
    },
    brokers: ['localhost:9098']
    //brokers: ['b-3.mskcluster.15hk8u.c8.kafka.eu-central-1.amazonaws.com:9098']
}
const kafka = new Kafka(kafkaConfig)
const admin = kafka.admin()

If I run this script remotely on the EC2 host by using the "actual" broker URL, it works fine and I can operate using the admin client. If I run it locally however, I always get this error:

Hostname verification failed","retryCount":0,"retryTime":271}
/Users/user/WebstormProjects/smi-dev-test-node/node_modules/kafkajs/src/protocol/requests/saslAuthenticate/v0/response.js:43
    throw new KafkaJSProtocolError({
          ^
KafkaJSProtocolError: [51c04b19-ba25-4499-a57f-622565fceac1]: Hostname verification failed
    at Object.parse (/Users/user/WebstormProjects/smi-dev-test-node/node_modules/kafkajs/src/protocol/requests/saslAuthenticate/v0/response.js:43:11)
    at Connection.send (/Users/user/WebstormProjects/smi-dev-test-node/node_modules/kafkajs/src/network/connection.js:433:35)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async saslAuthenticate (/Users/user/WebstormProjects/smi-dev-test-node/node_modules/kafkajs/src/broker/saslAuthenticator/index.js:45:30) {
  retriable: false,
  helpUrl: 'https://kafka.js.org/docs/configuration#sasl',
  type: 'SASL_AUTHENTICATION_FAILED',
  code: 58,
  [cause]: undefined

I wanted to use checkServerIdentity to circumvent this, but that does not work.

Expected behavior I would expect this callback to ignore hostname checks:

    ssl: {
        checkServerIdentity: () => undefined
    }

Environment: