KMinion is a feature-rich Prometheus exporter for Apache Kafka written in Go. It is lightweight and highly configurable so that it will meet your requirements.
MIT License
601
stars
123
forks
source link
Collect consumer groups data can't be disabled #243
Based on the info from documentation, I assume it shouldn't try to get consumer groups from brokers. But it actually trying:
Kminion log example:
{"level":"warn","ts":"2024-02-08T12:56:07.910Z","logger":"main.minion_service","msg":"failed to fetch consumer group offsets, inner kafka error","consumer_group":"SOME_NON_KMINION_GROUP_NAME","error":"failed to request group offsets for group 'SOME_NON_KMINION_GROUP_NAME': GROUP_AUTHORIZATION_FAILED: Not authorized to access group: Group authorization failed."}
Kafka auth log:
{"short_message":"Principal = User:kminion is Denied Operation = Describe from host = EDITED on resource = Group:LITERAL:SOME_NON_KMINION_GROUP_NAME","full_message":"Principal = User:kminion is Denied Operation = Describe from host = EDITED on resource = Group:LITERAL:SOME_NON_KMINION_GROUP_NAME","timestamp":1.707396472924E9,"level":6,"facility":"logstash-gelf","LoggerName":"kafka.authorizer.logger","SourceSimpleClassName":"SimpleAclAuthorizer$BaseAuthorizer","SourceClassName":"kafka.security.auth.SimpleAclAuthorizer$BaseAuthorizer","Time":"2024-02-08 12:47:52,924","Severity":"INFO","SourceLineNumber":66,"Thread":"data-plane-kafka-request-handler-0","SourceMethodName":"logAuditMessage"}
I suspect this is due to collectConsumerGroupLags function not having enabled check, like e.g. in collectConsumerGroups .
I've also tried using ignoredGroups: [ ".*" ], but since offset request happens earlier this doesn't have any effect.
Hi, I'd like to use Kminion only for E2E monitoring. So my config looks like this:
Based on the info from documentation, I assume it shouldn't try to get consumer groups from brokers. But it actually trying: Kminion log example:
Kafka auth log:
I suspect this is due to collectConsumerGroupLags function not having enabled check, like e.g. in collectConsumerGroups . I've also tried using
ignoredGroups: [ ".*" ]
, but since offset request happens earlier this doesn't have any effect.