aws / karpenter-provider-aws

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
https://karpenter.sh
Apache License 2.0
6.84k stars 963 forks source link

Karpenter metrics not working in Datadog after v1.0.0 migration. #6822

Closed tsiq-sanjanar closed 3 months ago

tsiq-sanjanar commented 3 months ago

Description

Observed Behavior: Karpenter metrics in Datadog worked until v0.37.0 but after migration it stopped working.

Expected Behavior: Karpenter metrics in datadog.

Reproduction Steps (Please include YAML): Steps followed https://docs.datadoghq.com/integrations/karpenter/

Versions:

tsiq-sanjanar commented 3 months ago

I found the issue. In the latest Karpenter version metrics port is updated to 8080 from 8000. Datadog documentation has not updated this change. Helm/Binary: controller.METRICS_PORT default changed back to 8080

From documentation https://docs.datadoghq.com/integrations/karpenter/ Updating below fixed the issue.

"openmetrics_endpoint": "http://%%host%%:8080/metrics"
jan-ludvik commented 2 months ago

That is one of the problems but many metrics were also renamed and they will not work until this PR is released with Datadog agent: https://github.com/DataDog/integrations-core/pull/18448