Closed geerlingguy closed 4 years ago
I think this is because the master has a taint to not execute workloads. I'll look on how to overcome this for this pod.
Yeah; I'm just wondering how node-exporter
overcomes that and ensures it's also running on the master, even though it has the same taint.
Supposedly this is the original fix for node-exporter-daemonset: https://github.com/coreos/prometheus-operator/pull/610/files
However, looking at the current version that's deployed I see:
tolerations:
- operator: Exists
It looks like that change is from this commit: https://github.com/coreos/kube-prometheus/commit/e4ff0f874638c1ec27f4f7b48b88b526045ebdf1#diff-22d9db260fb0516976f57a293971723d — generated from https://github.com/coreos/kube-prometheus/blob/5b9341cad63a30f8d1d1e008eccdc93f371caab3/jsonnet/kube-prometheus/node-exporter/node-exporter.libsonnet#L80
It looks like that change came from: https://github.com/coreos/kube-prometheus/commit/272ff23cb68f9fb06411cc316ca37fd8176ccff5
So maybe that's all that's required?
I'm currently doing a cluster rebuild, and I'll test this toleration once it's back up and running. In Pi time, though, that'll be an hour or so :D
Adding:
spec:
tolerations:
- operator: Exists
Worked! The DS is on all 7 nodes now, just like the node-exporter
. I'll work on a PR momentarily.
I noticed while debugging #39 that the
arm-exporter
DaemonSet was only running on 6 out of 7 Pi nodes. It was not running on the master node.The master has the following taint:
But that doesn't seem to cause the
node-exporter
DaemonSet to not deploy a Pod there:The arch is
arm
on all 7 Pis, so I'm not sure why the selector might influence the DS deployment.