search

danielqsj / kafka_exporter

Kafka exporter for Prometheus
Apache License 2.0
2.18k stars 610 forks source link

How to limit kafka exporter metrics #304

Open zarafree opened 2 years ago

zarafree commented 2 years ago

We wanted to gather only limited metrics from kafka exporter (Not JMX kafka metrics). For example, we need only kafka_consumergroup_current_offset and kafka_consumergroup_lag metrics from kafka exporter. We are getting OOMkilled for kafka exporter. We tried to increase pod limit but no luck so wanted to filter metrics so it will improve performance. any help / suggestions?

andyast commented 2 years ago

how many topics/partitions are you running? We have ~1400 topics with ~7500 partitions and kafka-exporter is using 75MB of ram.

See this response: https://github.com/danielqsj/kafka_exporter/issues/305#issuecomment-1162214186

msartor92 commented 1 year ago

It's not a ram problem but a scrape time problem

trycrmr commented 1 year ago

+1

devopsteam605 commented 1 year ago

+1

ksemele commented 2 months ago

Same for me... I set up a workaround with increased scrapeTimeout and interval in the Helm chart:

    prometheus:
      serviceMonitor:
        enabled: true
        scrapeTimeout: 4m
        interval: 5m

But this is not a good solution...