Open tommyJimmy87 opened 5 years ago
efeg
commented
5 years ago @tommyJimmy87 Thanks for reporting the issue!
Cruise Control does not support SASL_SSL, yet -- i.e. it currently supports SSL and Plaintext.
Would you be interested in submitting a patch to add SASL_SSL support?
becketqin
commented
5 years ago @tommyJimmy87 Are you trying to use Kerberos for authentication? If so, one of the following configs is needed:
sasl.kerberos.service.namesasl.jaas.configIt would be more helpful if you can share the configurations you are using.
tommyJimmy87
commented
5 years ago @becketqin We are not using kerberos... we are just using SASL_SSL with the username and password in the JAAS.. but apparently as @efeg said is not supported.
tommyJimmy87
commented
5 years ago @efeg I can take a look how complicate is to patch it.
becketqin
commented
5 years ago @tommyJimmy87 Hmm, CC should support all the mechanisms support by Kafka itself. Have you set sasl.mechanism in your config? By default it is going to be GSSAPI and kerberos will be used. With SASL_SSL, change the that config to PLAIN and have a try.
becketqin
commented
5 years ago Also, please make sure you are using the org.apache.kafka.common.security.plain.PlainLoginModule with your username and password in the jaas config file.
tommyJimmy87
commented
5 years ago @becketqin Thx, here my config file :
broker.metric.sample.store.topic=__KafkaCruiseControlModelTrainingSamples
sample.store.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore
ssl.key.password=xxxxxxxxx (password is present)
min.samples.per.broker.metrics.window=1
num.partition.metrics.windows=1
ssl.keystore.password=xxxxxxxxx (password is present)
sample.store.topic.replication.factor=2
ssl.keystore.location=/etc/kafka/secrets/jks/kafka.keystore.jks
capacity.config.file=/etc/kafka-cruise-control/config/capacity.json
webserver.ui.diskpath=/app/cruise-control-ui/dist/
zookeeper.connect=ZKIP:2181/devkafka
partition.metrics.window.ms=300000
metric.sampler.partition.assignor.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.DefaultMetricSamplerPartitionAssignor
ssl.truststore.location=/etc/kafka/secrets/jks/kafka.truststore.jks
ssl.client.auth=true
partition.metric.sample.store.topic=__KafkaCruiseControlPartitionMetricSamples
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
producer.security.protocol=SASL_SSL
client.id=kafka-cruise-control
broker.metrics.window.ms=300000
default.goals=com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal
producer.sasl.mechanism=PLAIN
bootstrap.servers=SASL_SSL://myIP:9094
cluster.configs.file=/etc/kafka-cruise-control/config/clusterConfigs.json
num.sample.loading.threads=8
num.broker.metrics.windows=20
ssl.truststore.password=xxxxxxxxx (password is present)
ssl.endpoint.identification.algorithm=
metric.sampler.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler
consumer.sasl.mechanism=PLAIN
consumer.security.protocol=SASL_SSL
num.metric.fetcher=1
auto.create.topics.enable=true
metric.sampling.interval.ms=120000
metric.reporter.topic.pattern=__CruiseControlMetricsAnd also I'm providing my JAAS file path in the KAFKA_OPTS env variable that should be added when CC runs. We are using the same JAAS file for several application.
tommyJimmy87
commented
5 years ago @becketqin The problem is just the KafkaAdminClient Configs, the ConsumerConf for example are created with the right values (jks.. password and jaas are present in the ConsumerConf but not in the KafkaAdminClient).
tommyJimmy87
commented
5 years ago @becketqin Uhm.. with the tag 0.1.39 is working actually.. earlier I was using 2.0.36 Tag... what's the different ? Master branch should be at the 0.1.39 right ? I guess it was a little bit confusing.
becketqin
commented
5 years ago @tommyJimmy87 Thanks for the update. This sounds like a bug in 2.0.36. But I could be wrong. @efeg would be more familiar with the diff between those two versions.
efeg
commented
5 years ago @tommyJimmy87 The reason why the tag 0.1.39 is working, but 2.0.36 is not is because former versions (i.e. 0.1.*) do not call describeLogDirs to show logDirs in the response of kafka_cluster_state (i.e. the endpoint that your request is sent to). The describeLogDirs creates and uses a KafkaAdminClient to return the corresponding response (supported for Kafka: 1.1.0+ with Cruise Control: 2.*).
To support SASL_SSL in 2.*, I believe there are (at least) two places to update (and test) -- i.e. you may search for the string if (securityProtocol.equals(SecurityProtocol.SSL.name)) to find the exact locations:
^ So I suspect that all we need would be:
else if (securityProtocol.equals(SecurityProtocol.SASL_SSL.name)) {
setStringConfigIfExists(configs, adminClientConfigs, SaslConfigs.SASL_JAAS_CONFIG);
setPasswordConfigIfExists(configs, adminClientConfigs, SaslConfigs.*); // Needs update for any other relevant password configs
setStringConfigIfExists(configs, adminClientConfigs, SaslConfigs.*); // Needs update for any other relevant string configs.
}@efeg Ok as I thought. If you want I can contribute on this.
efeg
commented
5 years ago @tommyJimmy87 Sure, we appreciate contributions!
Sugaroverdose
commented
5 years ago I've encountered the same issue with SASL_PLAINTEXT and sasl.mechanism=SCRAM-SHA-256: Cruise Control does connects to the cluster but right after that it tries to start AdminClient which does not inherit client.id, jaas.config and sasl.mechanism
UPD: managed to fix this by adding
import org.apache.kafka.common.config.SaslConfigs;
and
setStringConfigIfExists(configs, adminClientConfigs, SaslConfigs.SASL_MECHANISM);
into KafkaCruiseControlUtils.java
i guess the same should be added into CruiseControlMetricsUtils.java
efeg
commented
5 years ago @Sugaroverdose Sounds great! Would you consider creating a PR with the patch?
Sugaroverdose
commented
5 years ago @efeg i'll try to it seems like that CruiseControlMetricsUtils.java does not inherit securityProtocol, so idk how it works with something different than plaintext
Sugaroverdose
commented
5 years ago @tommyJimmy87 you may try latest migrate_to_kafka_2_0 branch, it should work in your case now
poyyi
commented
5 years ago I'm novice in adopting to cruise-contol using 2.029,trying with Sasl_ssl going by above discussion it looks I'm may not be able use it sasl_ssl could you suggest which release shld I pick .. Thank u
tommyJimmy87
commented
5 years ago @poyyi What is your Cruise Control Metric Jar configuration?
poyyi
commented
5 years ago @tommy it is cruise-control-metrics-reporter-0.1.0-SNAPSHOT.jar META-INF/ META-INF/MANIFEST.MF com/ com/linkedin/ com/linkedin/kafka/ com/linkedin/kafka/cruisecontrol/ com/linkedin/kafka/cruisecontrol/metricsreporter/ com/linkedin/kafka/cruisecontrol/metricsreporter/CruiseControlMetricsReporter$1.class com/linkedin/kafka/cruisecontrol/metricsreporter/CruiseControlMetricsReporter.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/ com/linkedin/kafka/cruisecontrol/metricsreporter/metric/YammerMetricProcessor$Context.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/YammerMetricProcessor.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/CruiseControlMetric$MetricClassId.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/CruiseControlMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/RawMetricType$MetricScope.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/RawMetricType.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/BrokerMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/MetricSerde$1.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/MetricSerde.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/MetricsUtils.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/PartitionMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/TopicMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/CruiseControlMetricsReporterConfig.class com/linkedin/kafka/cruisecontrol/metricsreporter/exception/
on the kafka servers.. "2024" is sasl_ssl port metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter cruise.control.metrics.reporter.bootstrap.servers=0.0.0.0:2024 auto.create.topics.enable=true
CC property file bootstrap.servers=BK1-IP:2024,BK2-IP:2024,BK3-IP:2024,BK4-IP:2024,BK5-IP:2024 zookeeper.connect=ZK1-IP:2181,ZK2-IP:2181,ZK3-IP:2181 sample.store.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore ssl.key.password=XXXXX@123 ssl.keystore.password=XXXXX@123 ssl.keystore.location=/opt/app/work/kafka-0.10.01/certs/kafka.server.truststore.jks ssl.truststore.location=/opt/app/work/kafka-0.10.01/certs/kafka.server.truststore.jks ssl.client.auth=true sasl.mechanism=PLAIN security.protocol=SASL_SSL producer.security.protocol=SASL_SSL producer.sasl.mechanism=PLAIN ssl.truststore.password=XXXXX@123 ssl.endpoint.identification.algorithm= consumer.sasl.mechanism=PLAIN consumer.security.protocol=SASL_SSL
tommyJimmy87
commented
5 years ago I guess you are missing all the Cruise Control configs for Sasl SSL.
Try with these also :
KAFKA_METRIC_REPORTERS: "com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SECURITY_PROTOCOL: "SASL_SSL"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SASL_MECHANISM: "PLAIN"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: ""
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_CLIENT_AUTH: "true"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_KEYSTORE_LOCATION: '/path/to/kafka.server.keystore.jks'
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_TRUSTSTORE_LOCATION: '/path/to/kafka.server.truststore.jks'
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SASL_MECHANISM_INTER_BROKER_PROTOCOL: "{{.kafkaSaslMechanismInterBrokerProtocol}}"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SASL_ENABLED_MECHANISMS: "{{.kafkaSaslEnabledMechanisms}}"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:SASL_SSL,CLIENT:SASL_SSL,BROKER:SASL_SSL,PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_SSL:SASL_SSL"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:SSL,CLIENT:SSL,BROKER:SSL,PLAINTEXT:PLAINTEXT,SSL:SSL"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:PLAINTEXT,CLIENT:PLAINTEXT,BROKER:PLAINTEXT,PLAINTEXT:PLAINTEXT,SSL:SSL"
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_ADVERTISED_LISTENERS={advertised_listeners_list}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_TRUSTSTORE_PASSWORD={trustStorePass}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_KEYSTORE_PASSWORD={keyStorePass}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_KEY_PASSWORD={keyPass}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_BOOTSTRAP_SERVERS={yoursBootstrapServer}@jimmy sorry could not try immly on SASL_SSL (but tired with SSL which i run into some other issue described below ).. i have few queries on your suggestions for SASL_SSL
1)Above mentioned properties shld go into server.properties on brokers? 2) shld they be given exactly like how you gave above KAFKA_METRIC_REPORTERS: ?, currently i have it as below in my server.properties metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter cruise.control.metrics.reporter.security.protocol=SSL
PL note version I'm using: Cruise control version: Cruise-control-2.0.42 Kafka :kafka_2.12-0.11.0.0 (Kafka version : 0.11.0.1)
I ran with SSL pl find my setting and Exception I'm getting for
Failures "GET /kafkacruisecontrol/kafka_cluster_state?json=true HTTP/1.1" 500 4462 79
Success "GET /kafkacruisecontrol/user_tasks?json=true HTTP/1.1" 200 2637 8 "GET /kafkacruisecontrol/state?substates=EXECUTOR&verbose=true&json=true HTTP/1.1" 200 61 2 "GET /kafkacruisecontrol/state?substates=MONITOR&verbose=true&json=true HTTP/1.1" 200 288 1 "GET /kafkacruisecontrol/state?substates=ANALYZER&verbose=true&json=true HTTP/1.1" 200 2866 2 "GET /kafkacruisecontrol/load?allow_capacity_estimation=true&json=true HTTP/1.1" 200 3074 2 "GET /kafkacruisecontrol/user_tasks?json=true HTTP/1.1" 200 3075 2 "GET /kafkacruisecontrol/state?substates=EXECUTOR&verbose=true&json=true HTTP/1.1" 200 61 1 "GET /kafkacruisecontrol/user_tasks?json=true HTTP/1.1" 200 3275 2
Exception: GET request '/kafka_cluster_state' due to: 'Failed to populate broker logDir state.'. (com.linkedin.kafka.cruisecontrol.servlet.KafkaCruiseControlServlet) java.lang.RuntimeException: Failed to populate broker logDir state. at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.getJSONString(KafkaClusterState.java:90) at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.discardIrrelevantAndCacheRelevant(KafkaClusterState.java:456) at com.linkedin.kafka.cruisecontrol.servlet.response.AbstractCruiseControlResponse.discardIrrelevantResponse(AbstractCruiseControlResponse.java:49) Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_LOG_DIRS ---------------Cruisecontrol.properties Setting---- Cruise control.properties security.protocol=SSL ssl.truststore.type=JKS ssl.truststore.location=/opt/app/workload/cruise-control-2.0.42/config/certs/kafka.server.truststore.jks ssl.truststore.password=pxxxxxxxx@123 ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
---------Broker server.properties---------------- listeners=PLAINTEXT://bk1:2021,SASL_PLAINTEXT://bk1:2022,SSL://bk1:2023,SASL_SSL://bk1:2024 ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 security.inter.broker.protocol=SASL_SSL sasl.mechanism.inter.broker.protocol=PLAIN sasl.enabled.mechanisms=PLAIN super.users=User:mxxxx allow.everyone.if.no.acl.found=true authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
advertised.listeners=PLAINTEXT://bk1:2021,SASL_PLAINTEXT://bk1:2022,SSL://bk1:2023,SASL_SSL://bk1:2024 ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
############################# Log Basics #############################
log.dirs=/opt/app/workload/enabler/kafka_2.12-0.11.0.0/kafka-logs
group.initial.rebalance.delay.ms=0 metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter cruise.control.metrics.reporter.security.protocol=SSL cruise.control.metrics.reporter.bootstrap.servers=0.0.0.0:2023 auto.create.topics.enable=true cruise.control.metrics.reporter.ssl.truststore.location=/opt/app/workload/kafka_2.12-0.11.0.0/config/certs/kafka.server.truststore.jks cruise.control.metrics.reporter.ssl.truststore.password=pxxxxxxxx@123 cruise.control.metrics.reporter.ssl.keystore.location=/opt/app/workload/kafka_2.12-0.11.0.0/config/certs/kafka.server.keystore.jks cruise.control.metrics.reporter.ssl.keystore.password=pxxxxxxxx@123 cruise.control.metrics.reporter.ssl.key.password=pxxxxxxxx@123 broker.rack=Rack1
poyyi
commented
5 years ago @tommyJimmy87 ...did you get chance to look at my query
ishita07-danaher
commented
4 years ago I am facing the same issue. Can you please update on this ?
Arullaldivakar
commented
3 years ago Does cruise control support TLS SASL/Scram enabled Kafka secure cluster?
HirossxD
commented
3 months ago I have the same issue using SASL_PLAINTEXT, CC not working on kerberized kafka cluster,
i am using older commit of this repo: "repo_commit": "6e8f2ef0209903dc1036857edff1e1020058f02e"
seems like AdminClientConfig does not fetch/load sasl.kereberos.service.name , it is null even if specified in cruise_control_jaas.conf and cruisecontrol.properties files (or kafka_opts)
[2024-07-23 04:29:52,734] INFO COMMIT INFO: 2.5.126---6e8f2ef0209903dc1036857edff1e1020058f02e (com.linkedin.kafka.cruisecontrol.KafkaCruiseControl)
[2024-07-23 04:29:52,744] INFO AdminClientConfig values:
bootstrap.servers = [poc-brkfk-1:9092, poc-brkfk-2:9092, poc-brkfk-3:9092]
client.dns.lookup = use_all_dns_ips
client.id =
connections.max.idle.ms = 300000
default.api.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 50
reconnect.backoff.ms = 50
request.timeout.ms = 180000
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.connect.timeout.ms = null
sasl.login.read.timeout.ms = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.login.retry.backoff.max.ms = 10000
sasl.login.retry.backoff.ms = 100
sasl.mechanism = GSSAPI
sasl.oauthbearer.clock.skew.seconds = 30
sasl.oauthbearer.expected.audience = null
sasl.oauthbearer.expected.issuer = null
sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
sasl.oauthbearer.jwks.endpoint.url = null
sasl.oauthbearer.scope.claim.name = scope
sasl.oauthbearer.sub.claim.name = sub
sasl.oauthbearer.token.endpoint.url = null
security.protocol = SASL_PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
socket.connection.setup.timeout.max.ms = 30000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.3
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
(org.apache.kafka.clients.admin.AdminClientConfig)
[2024-07-23 04:29:52,781] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
[2024-07-23 04:29:52,783] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
[2024-07-23 04:29:52,783] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
[2024-07-23 04:29:52,783] ERROR Uncaught exception on thread Thread[main,5,main] (com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain)
org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:538) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.clients.admin.Admin.create(Admin.java:143) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49) ~[kafka-clients-3.1.0.jar:?]
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlUtils.createAdminClient(KafkaCruiseControlUtils.java:582) ~[cruise-control-2.5.126.jar:?]
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.<init>(KafkaCruiseControl.java:120) ~[cruise-control-2.5.126.jar:?]
at com.linkedin.kafka.cruisecontrol.async.AsyncKafkaCruiseControl.<init>(AsyncKafkaCruiseControl.java:34) ~[cruise-control-2.5.126.jar:?]
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlApp.<init>(KafkaCruiseControlApp.java:36) ~[cruise-control-2.5.126.jar:?]
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlServletApp.<init>(KafkaCruiseControlServletApp.java:32) ~[cruise-control-2.5.126.jar:?]
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlUtils.getCruiseControlApp(KafkaCruiseControlUtils.java:920) ~[cruise-control-2.5.126.jar:?]
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.main(KafkaCruiseControlMain.java:37) ~[cruise-control-2.5.126.jar:?]
Caused by: org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: No serviceName defined in either JAAS or Kafka config
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:509) ~[kafka-clients-3.1.0.jar:?]
... 9 more
Caused by: java.lang.IllegalArgumentException: No serviceName defined in either JAAS or Kafka config
at org.apache.kafka.common.security.kerberos.KerberosLogin.getServiceName(KerberosLogin.java:303) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.security.kerberos.KerberosLogin.configure(KerberosLogin.java:92) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:61) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:105) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105) ~[kafka-clients-3.1.0.jar:?]
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:509) ~[kafka-clients-3.1.0.jar:?]
... 9 morei will try same procedure on the latest commit, will see if issue persist.
HirossxD
commented
3 months ago switched to commit f23332a2ec4f8f3f0d69cd6e6f3b7aa9e33fb873 and cruisecontrol started without an issue..
here are some necessary configuration of cruisecontrol.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
cruise.control.metrics.reporter.security.protocol=SASL_PLAINTEXT
cruise.control.metrics.reporter.sasl.mechanism=GSSAPI
cruise.control.metrics.reporter.sasl.kerberos.service.name=kafka
# Kerberos principal and keytab for Cruise Control
cruise.control.metrics.reporter.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
useKeyTab=true \
storeKey=true \
keyTab="/etc/security/keytabs/kafka.service.keytab" \
principal="kafka/poc-cc-vm@KAFKA.POC.LOCAL";
# Kerberos configuration file location
java.security.auth.login.config=/opt/cruise-control/config/cruise_control_jaas.conf
java.security.krb5.conf=/etc/krb5.conf
but cruisecontrol logs says
INFO Collected 1 broker metric samples for 1 brokers. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingFetcher)i have just setup cc parameters only on one broker yet.
current broker configuration lines related to CC and kerberos are
metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter
cruise.control.metrics.topic.auto.create=true
cruise.control.metrics.topic.num.partitions=1
cruise.control.metrics.topic.replication.factor=1
cruise.control.metrics.reporter.bootstrap.servers=poc-brkfk-1:9092
cruise.control.metrics.reporter.security.protocol=SASL_PLAINTEXT
cruise.control.metrics.reporter.sasl.mechanism=GSSAPI
cruise.control.metrics.reporter.sasl.kerberos.service.name=kafka
# Kerberos principal and keytab for Cruise Control
cruise.control.metrics.reporter.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
useKeyTab=true \
storeKey=true \
keyTab="/etc/security/keytabs/kafka.service.keytab" \
principal="kafka/poc-brkfk-1@KAFKA.POC.LOCAL";so it is functional i guess.
I'm using Security Protocol set to "SASL_SSL" which means I need to specify all the JKS and JAAS in order to connect to the Cluster. I'm providing those information through the property file, but I found out that when the KafkaAdminClient runs it gets wrong SSL configuration because of this :
I'm not using
SSLbutSASL_SSLand I still need those properties to be set in the KafkaClientAdmin configurations.The class where i found that code portion is KafkaCruiseControlUtils.java.
This is the error I'm facing :
Tag Cruise Control version is 2.0.36. Kafka Version is 1.1.x.