Closed benbc closed 6 years ago
Sorry for spamming you today. :-)
Hi @benbc thanks for the report, it seems that the KSM project recently added the 8081 port in the exposed port list and this messing up with the autodiscovery template, because in case of multiple exposed port, we'll take the highest port number. I've made a PR to fix the official AD template https://github.com/DataDog/integrations-core/pull/1308 in the meanwhile you can either edit the AD template or try using annotations on the KSM pod
@mfpierre Thank you.
I'm sure I can use the annotation approach. The diff for your fix will help with that. I was unsure, reading your docs, which pod the annotation should go on (datadog-agent
or kube-state-metrics
), but your comment above makes that clear, so I now I have everything I need.
What is the easiest way for me to trace your linked fix in https://github.com/DataDog/integrations-core to a published version of the datadog/datadog-agent
Docker image?
@benbc just merged the PR should go out with the next agent release
Here is the proper way to fix this issue with annotations for Datadog agent version 5:
diff --git a/kube-state-metrics/kube-state-metrics-deployment.yaml b/kube-state-metrics/kube-state-metrics-deployment.yaml
index 2e8dc47..92417e4 100644
--- a/kube-state-metrics/kube-state-metrics-deployment.yaml
+++ b/kube-state-metrics/kube-state-metrics-deployment.yaml
@@ -14,6 +14,10 @@ spec:
metadata:
labels:
k8s-app: kube-state-metrics
+ annotations:
+ service-discovery.datadoghq.com/kube-state-metrics.check_names: '["kubernetes_state"]'
+ service-discovery.datadoghq.com/kube-state-metrics.init_configs: '[{}]'
+ service-discovery.datadoghq.com/kube-state-metrics.instances: '[{"kube_state_url": "http://%%host%%:%%port_0%%/metrics"}]'
spec:
serviceAccountName: kube-state-metrics
containers:
Datadog agent version 6 users should probably only need to replace service-discovery.datadoghq.com
by ad.datadoghq.com
.
Edit: note the use of %%port_0%%
instead of a hard coded value.
I'm using the DataDog agent (v6.1.0) and
kube-state-metrics
(v1.2.0) on GKE (v1.8.9). I'm using the Kubernetes manifests provided by both projects with minimal modifications.I see this error in my logs:
It looks like the check is trying to scrape the
/metrics
URL from port 8081. Howeverkube-state-metrics
exposes that URL on port 8080 by default. (The IP address of the container is correct.)I've spent a while reading your docs and spelunking in a couple of your codebases and I can't work out how that default gets set or how to override it.
So I suppose I have a couple of questions:
Thanks -Ben