confluentinc / jmx-monitoring-stacks

📊 Monitoring examples for Confluent Cloud and Confluent Platform
Apache License 2.0
59 stars 170 forks source link

Add JMX Exporter Configs for Producers within the Confluent Server #62

Open nerdynick opened 2 years ago

nerdynick commented 2 years ago

When Control Center is enabled we leverage the ConfluentMetricsReporter within the Brokers. We should capture these metrics into prometheus.

Example Configs:

- pattern: kafka.producer<type=producer-metrics, client-id=(.+)><>(.+):\w* 
   name: kafka_producer_$2 
   labels: 
     client-id: "$1"
- pattern: kafka.producer<type=producer-node-metrics, client-id=(.+), node-id=(.+)><>(.+):\w* 
   name: kafka_producer_$3 
   labels: 
     client-id: "$1" 
     broker: "$2"
- pattern: kafka.producer<type=producer-topic-metrics, client-id=(.+), topic=(.+)><>(.+):\w* 
   name: kafka_producer_$3 
   labels: 
     client-id: "$1" 
     topic: "$2"
waliaabhishek commented 2 years ago

SOme of those were deliberately blacklisted due to the sheer volume that they start producing at scale. Some folks are pretty sensitive to the scrape performance and that caused a bit of pushback. I can try it out again for the 7.x bump and see if it is okay with minimal scrape times.