DataDog / integrations-core

Core integrations of the Datadog Agent
BSD 3-Clause "New" or "Revised" License
928 stars 1.4k forks source link

exclude_metrics of aws_msk in clusterAgent.confd.aws_msk.yaml doesn't work #17375

Open ellazhao-testo opened 6 months ago

ellazhao-testo commented 6 months ago

According to the aws msk conf.yaml.example, we can use exclude_metrics to exclude metrics sent to datadog, but it doesn't work.

here is the description for exclude_metrics:

## @param exclude_metrics - list of strings - optional
## A list of metrics to exclude, with each entry being either
## the exact metric name or a regular expression.
## In order to exclude all metrics but the ones matching a specific filter,
## you can use a negative lookahead regex like:
##   - ^(?!foo).*$
#
# exclude_metrics: []

Additional environment details (Operating System, Cloud provider, etc):

Steps to reproduce the issue:

  1. create a aws msk cluster and enable JMX and NodeExporter metrics in aws console
  2. config the datadog MSK intergration
  3. install datadog agent and set amazon_msk configuration on aws EKS cluster
  4. here is the msk configration in datadog-agent-values.yaml
    clusterAgent:
    confd:
        amazon_msk.yaml: |-
            cluster_check: true
            instances:
              - use_openmetrics: true
                cluster_arn: ${msk_cluster_arn}
                exclude_metrics:
                  - ^.*go.*$

    Describe the results you received: all aws.msk.kafka.go metrics still sent to datadog

Describe the results you expected: All aws.msk.kafka.go metrics should not be sent to datadog

Additional information you deem important (e.g. issue happens only occasionally): issue happens always

ellazhao-testo commented 6 months ago

we reviewed integrations-core source code, the should_send_metric function only check the metric_patterns:exclude/include config in datadog_checks/base/checks/base.py.

we changed our configration to metric_patterns: include/exclude , now it worked.

so my question is: does exclude_metrics still in use? If yes, it is a bug, because it doesn't work . If no, conf.yamL.example seems outdated.