airbnb / kafka-statsd-metrics2

Send Kafka Metrics to StatsD.
Apache License 2.0
135 stars 49 forks source link

Kafka 0.8.2.2 throwing exception using release 0.4.1 #24

Open agolomoodysaada opened 7 years ago

agolomoodysaada commented 7 years ago

I added kafka.metrics.reporters=com.airbnb.kafka.kafka08.StatsdMetricsReporter to my consumer.properties file and i see this

$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test -from-beginning --consumer.config config/consumer.properties
[2017-07-14 21:31:25,614] WARN Property kafka.metrics.reporters is not valid (kafka.utils.VerifiableProperties)
Exception in thread "main" java.lang.ClassNotFoundException: com.airbnb.kafka.kafka08.StatsdMetricsReporter
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at kafka.utils.Utils$.createObject(Utils.scala:435)
    at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:59)
    at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:58)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35)
    at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:58)
    at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:140)
    at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:143)
    at kafka.consumer.Consumer$.create(ConsumerConnector.scala:94)
    at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:145)
    at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
treziac commented 7 years ago

Try using metric.reporters rather than kafka.metrics.reporters

aheinz-fe commented 7 years ago
$ jar -tf kafka-statsd-metrics2-0.4.1.jar 
META-INF/
META-INF/MANIFEST.MF
com/
com/airbnb/
com/airbnb/kafka/
com/airbnb/kafka/KafkaStatsdMetricsReporter.class
com/airbnb/kafka/KafkaStatsdMetricsReporterMBean.class
com/airbnb/metrics/
com/airbnb/metrics/Dimension.class
com/airbnb/metrics/ExcludeMetricPredicate.class
com/airbnb/metrics/MetricNameFormatter.class
com/airbnb/metrics/Parser.class
com/airbnb/metrics/ParserForNoTag.class
com/airbnb/metrics/ParserForTagInMBeanName.class
com/airbnb/metrics/StatsDReporter.class

suggests

kafka.metrics.reporters=com.airbnb.kafka.KafkaStatsdMetricsReporter

I'm using the same versions as you and this got me past the ClassNotFoundException, but I'm not seeing anything sent to statsd yet.

aheinz-fe commented 7 years ago

Got it partially working. Dropping java-dogstatsd-client-2.0.11.jar into libs/ and downgrading kafka to 0.8.2.1 (an exact match for 0.4.1/build.gradle#L26) seems to have most of the metrics showing up, minus the tag-requiring metrics mentioned in https://github.com/airbnb/kafka-statsd-metrics2/issues/9.