search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

Cruise Control cannot find sampling topic matches __CruiseControlMetrics #24

Closed wander3r closed 6 years ago

wander3r commented 7 years ago

i got this error when i try to start Cruise Control:

[2017-09-06 09:48:26,651] INFO Kafka version : 0.10.1.0 (org.apache.kafka.common.utils.AppInfoParser) [2017-09-06 09:48:26,651] INFO Kafka commitId : 3402a74efb23d1d4 (org.apache.kafka.common.utils.AppInfoParser) Exception in thread "main" java.lang.IllegalStateException: Cruise Control cannot find sampling topic matches __CruiseControlMetrics in the target cluster. at com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler.configure(CruiseControlMetricsReporterSampler.java:175) at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:207) at com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager.(MetricFetcherManager.java:110) at com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager.(MetricFetcherManager.java:62) at com.linkedin.kafka.cruisecontrol.monitor.task.LoadMonitorTaskRunner.(LoadMonitorTaskRunner.java:61) at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.(LoadMonitor.java:112) at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.(KafkaCruiseControl.java:75) at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.main(KafkaCruiseControlMain.java:47)

did i miss some configurations or should i create this topic manually?

becketqin commented 7 years ago

@wander3r Cruise Control Metrics Reporter will produce metrics to this topic. So if your Kafka cluster has auto.create.topics.enable set to true, there is no need to manually create this topic. Otherwise you may have to do so. We should probably let CruiseControlMetricsReporter to create this topic automatically, I'll file a new issue for this.

sharepusher commented 7 years ago

So cruise control do NOT create the topic automatically at present? I have to manually created __CruiseControlMetrics even I set enabled auto.create.topics.enable to True?

sharepusher commented 7 years ago

@becketqin I have enabled auto.create.topics.enable to true, and restarted the brokers, but why the __CruiseControlMetrics still not created automatically, I do NOT want to create the topic manually, what should I do to make it work? Thank you

sharepusher commented 7 years ago

does it mean, I have to compile the cruise control in my target broker? then the __CruiseControlMetrics will be automatically created automatically? I don't wanna manually create it as it will be wired and hard to maintain if I tell my colleagues to make it on their new clusters

sharepusher commented 7 years ago

@wander3r have u solved this problem? created it manually? thanks.

wander3r commented 7 years ago

@sharepusher yes, i ceated this topic manually.otherwsie, the "metric.reporter.topic.pattern" configuration confused me in the first place, i thought there would be more than one topic to create.

njpsanghani commented 6 years ago

I had the same issue, but was resolved by this. set enabled auto.create.topics.enable to True or manually create the topic does the trick.