For a self-hosted kafka cluster, only part of the list of groups is returned for each broker, and there is no duplication.
At azure event hub, for each request, the full list of groups is returned, as a result, the metrics for Prometheus are duplicated.
An error has occurred while serving metrics:
6030 error(s) occurred:
collected metric "kafka_consumergroup_members" { label:<name:"consumergroup" value:"delivery-*" > gauge: } was collected before with the same name and label values
collected metric "kafka_consumergroup_members" { label:<name:"consumergroup" value:"cashier-*" > gauge: } was collected before with the same name and label values
collected metric "kafka_consumergroup_members" { label:<name:"consumergroup" value:"delivery-*" > gauge: } was collected before with the same name and label values
Related to the same issue already the issue is going on and it was reported in git hub
When kafka_exporter do a RefreshMetadata request to get a list of brokers, it get 16 brokers in response that differ in CamelCase addresses
[we-eventhub-.servicebus.windows.net:9093] [We-eventhub-.servicebus.windows.net:9093 [WE-eventhub-.servicebus.windows.net:9093 [WE-Eventub-.servicebus.windows.net:9093] [WE-EVenthub-.servicebus.windows.net:9093] [WE-EVEnthub-.servicebus.windows.net:9093] [WE-EVENthub-.servicebus.windows.net:9093] [WE-EVENThub-.servicebus.windows.net:9093] [WE-EVENTHub-.servicebus.windows.net:9093] [WE-EVENTHUb-.servicebus.windows.net:9093] [WE-EVENTHUB-.servicebus.windows.net:9093] [WE-EVENTHUB-.servicebus.windows.net:9093] [WE-EVENTHUB-.servicebus.windows.net:9093] [WE-EVENTHUB-.servicebus.windows.net:9093] [WE-EVENTHUB-.servicebus.windows.net:9093] [WE-EVENTHUB-.Servicebus.windows.net:9093]
For every time it is increasing the upper case.
The line of code where the request happens: https://github.com/danielqsj/kafka_exporter/blob/cf1ca53416cb76e42c4cab5c98b9500205723804/vendor/github.com/Shopify/sarama/broker.go#L360
Next, kafka_exporter, in order to collect consumergroup metrics, makes a ListGroupsRequest request for each broker https://github.com/danielqsj/kafka_exporter/blob/102399e55a6bce688dd75880cd6d169e4b231101/kafka_exporter.go#L550
For a self-hosted kafka cluster, only part of the list of groups is returned for each broker, and there is no duplication.
At azure event hub, for each request, the full list of groups is returned, as a result, the metrics for Prometheus are duplicated.
An error has occurred while serving metrics:
6030 error(s) occurred:
Related to the same issue already the issue is going on and it was reported in git hub
https://github.com/danielqsj/kafka_exporter/issues/306
Concern is this a bug? or normal behavior? Will it be fixed?