search

yahoo / CMAK

CMAK is a tool for managing Apache Kafka clusters
Apache License 2.0
11.83k stars 2.51k forks source link

Set mechanism for kafka cluster #640

Open woshiduncan opened 5 years ago

woshiduncan commented 5 years ago

With 1.3.3.22,we can set different security.protocol and sasl.mechanism for different cluster in UI,but how to deal with the mechanism not GSSAPI . The ERROR is

org.apache.kafka.common.errors.IllegalSaslStateException: Unexpected handshake request with client mechanism SCRAM-SHA-256, enabled mechanisms are [GSSAPI]

It's not a question about mistake config in kafka , our server.properties already enable SCRAM-SHA-256 by sasl.enabled.mechanisms=GSSAPI,PLAIN,SCRAM-SHA-256,SCRAM-SHA-512

winter98 commented 5 years ago

my understanding was that kafka-manager couldn't handle SASL_SCRAM at the moment ? looks like SASL_PLAIN has been merged in : https://github.com/yahoo/kafka-manager/pull/632 but not sure anyone's implemented SCRAM yet ? I get this in my log (2.0.0.2 kafka-manager) :

2019-04-23 06:34:44,414 - [WARN] - from org.apache.kafka.clients.NetworkClient in KafkaTopicOffsetGetter [Consumer clientId=consumer-12, groupId=null] Error while fetching metadata with correlation id 74 : {testnested=TOPIC_AUTHORIZATION_FAILED}

even though there aren't ACLs on the topic __consumer_offsets and the session is etstablished :

2019-04-23 06:03:46,474 - [INFO] - from org.apache.zookeeper.Login in kafka-manager-system-akka.actor.default-dispatcher-2-SendThread(xxxxxxxxxx:2181) Client successfully logged in.

2019-04-23 06:03:46,478 - [INFO] - from org.apache.zookeeper.client.ZooKeeperSaslClient in kafka-manager-system-akka.actor.default-dispatcher-2-SendThread(xxxxxxxxxx:2181) Client will use DIGEST-MD5 as SASL mechanism.

jurgen-weber-deltatre commented 3 years ago

https://github.com/yahoo/CMAK/issues/828 related?

jurgen-weber-deltatre commented 3 years ago

SASL_SCRAM would be required to use this with AWS MSK, etc.

tanuj83 commented 2 years ago

Works for me It seems the root cause as CMAK somehow using 1st mechanism from sasl.enabled.mechanisms of Kafka broker to connect instead of using defined if multi protocols are enabled on Kafka. I changed the order in kafka broker config to define SCRAM as first in the order and CMAK started working for me

Previous - not working sasl.enabled.mechanisms=GSSAPI,SCRAM-SHA-512,OAUTHBEARER

Now - Working sasl.enabled.mechanisms=SCRAM-SHA-512,OAUTHBEARER,GSSAPI

similar I did for listener, security.protocol.map as well