search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

cannot find sampling topic matches __CruiseControlMetrics in the target cluster. #21

Closed spark8899 closed 7 years ago

spark8899 commented 7 years ago

[2017-09-04 13:37:11,377] INFO ConsumerConfig values: auto.commit.interval.ms = 5000 auto.offset.reset = latest bootstrap.servers = [172.31.217.202:9092, 172.31.217.203:9092, 172.31.217.204:9092] check.crcs = true client.id = CruiseControlMetricsReporterSamplerConsumer-1417835929 connections.max.idle.ms = 540000 enable.auto.commit = false exclude.internal.topics = true fetch.max.bytes = 52428800 fetch.max.wait.ms = 500 fetch.min.bytes = 1 group.id = CruiseControlMetricsReporterSampler-6370840339601264904 heartbeat.interval.ms = 3000 interceptor.classes = null key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer max.partition.fetch.bytes = 1048576 max.poll.interval.ms = 2147483647 max.poll.records = 2147483647 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.sample.window.ms = 30000 partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor] receive.buffer.bytes = 65536 reconnect.backoff.ms = 50 request.timeout.ms = 305000 retry.backoff.ms = 100 sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT send.buffer.bytes = 131072 session.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS value.deserializer = class com.linkedin.kafka.cruisecontrol.metricsreporter.metric.MetricSerde (org.apache.kafka.clients.consumer.ConsumerConfig) [2017-09-04 13:37:11,390] INFO Kafka version : 0.10.1.0 (org.apache.kafka.common.utils.AppInfoParser) [2017-09-04 13:37:11,391] INFO Kafka commitId : 3402a74efb23d1d4 (org.apache.kafka.common.utils.AppInfoParser) Exception in thread "main" java.lang.IllegalStateException: Cruise Control cannot find sampling topic matches __CruiseControlMetrics in the target cluster. at com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler.configure(CruiseControlMetricsReporterSampler.java:175) at com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager.(MetricFetcherManager.java:111) at com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager.(MetricFetcherManager.java:62) at com.linkedin.kafka.cruisecontrol.monitor.task.LoadMonitorTaskRunner.(LoadMonitorTaskRunner.java:61) at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.(LoadMonitor.java:112) at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.(KafkaCruiseControl.java:75) at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.main(KafkaCruiseControlMain.java:47)

My kafka version: kafka_2.11-0.11.0.0 kafka topice: __CruiseControlMetrics test

spark8899 commented 7 years ago

I use kafka_0.10.1.0 is ok. thk you.

becketqin commented 7 years ago

@spark8103 When you use the metrics reporter for the first time, it may take some time before the metrics topic is created. When Cruise Control starts and could not find the topic, it shutdown by itself. So if you wait a little longer to let the CruiseControlMetricsReporter create the topic, then start Cruise Control, it should work. That said, we should probably let Cruise Control wait for some time if the metrics topic is not found.