search

grepplabs / kafka-proxy

Proxy connections to Kafka cluster. Connect through SOCKS Proxy, HTTP Proxy or to cluster running in Kubernetes.
Apache License 2.0
501 stars 87 forks source link

MSK Serverless Net Address Not Found #175

Open srikanthvpai opened 4 months ago

srikanthvpai commented 4 months ago

Hi All,

I am able to successfully hit a MSK provisioned cluster, but not for serverless. Any idea as to why this might be the case ? I am getting the below net address not found error. INFO[2024-07-31T22:04:52Z] New connection for boot-<<>>.c2.kafka-serverless.us-east-1.amazonaws.com:9098 DEBU[2024-07-31T22:04:52Z] Successful IAM SASL handshake. Available mechanisms: [OAUTHBEARER AWS_MSK_IAM] DEBU[2024-07-31T22:04:52Z] Kafka request key 18, version 3, length 46 DEBU[2024-07-31T22:04:52Z] Kafka response key 18, version 3, length 253 DEBU[2024-07-31T22:04:52Z] Kafka request key 3, version 11, length 56 DEBU[2024-07-31T22:04:52Z] Kafka response key 3, version 11, length 3852 INFO[2024-07-31T22:04:52Z] Reading data from boot-<<>>.c2.kafka-serverless.us-east-1.amazonaws.com:9098 had error: net address mapping for b97-<<>>.c2.kafka-serverless.us-east-1.amazonaws.com:9098 was not found

Not sure why the boot is getting replaced by b97 while its trying to hit the broker url. The machine on which this proxy container or process has a role with complete access to that MSK cluster and I can run kafka commands on that.

Command used to start the proxy server :

kafka-proxy server --bootstrap-server-mapping "boot-<<>>.c2.kafka-serverless.us-east-1.amazonaws.com:9098,0.0.0.0:30001,<>:<>" --external-server-mapping "boot-<<>>.c2.kafka-serverless.us-east-1.amazonaws.com:9098,<>:<>" --dynamic-listeners-disable --tls-enable --tls-insecure-skip-verify --sasl-enable --sasl-method "AWS_MSK_IAM" --sasl-aws-region "us-east-1" --log-level debug

Regards

shalevamit9 commented 3 months ago

Hey @srikanthvpai, did you have progress with this? I'm having the same issue.