Closed elldritch closed 1 year ago
Hi @elldritch
Thanks for opening this issue, we are starting our investigation.
In the meantime, could you share which agent version you are using? 🙇
same problem here, our cluster agent's version is:
Cluster Agent 7.44.1 - Commit: 299bdcd - Serialization version: v5.0.76 - Go version: go1.19.8
Describe what happened:
We upgraded to Datadog 3.30.5 this morning and experienced an issue where
kubernetes_state.*
metrics suddenly stopped working. In thekubectl logs
of thecluster-agent
pod, we found:We suspect that this is related to a change that occurred in https://github.com/DataDog/helm-charts/compare/datadog-3.30.4...datadog-3.30.5, specifically https://github.com/DataDog/helm-charts/pull/1055.
We were able to resolve this issue by downgrading to 3.30.4, although we weren't able to root cause this error.
Describe what you expected:
I expected the
kubernetes_state.*
metrics to continue to work.Steps to reproduce the issue:
From 3.30.4, we ran
helm upgrade datadog datadog/datadog --reuse-values
.Additional environment details (Operating System, Cloud provider, etc):
We run a self-hosted Kubernetes cluster on top of AWS EC2 instances.