helm / charts

⚠️(OBSOLETE) Curated applications for Kubernetes
Apache License 2.0
15.49k stars 16.8k forks source link

stable/promethues-operator kubelet alert still firing while disabled #18799

Closed acarsercan closed 4 years ago

acarsercan commented 4 years ago

Hi,

Running prometheus-operator-7.4.0 0.34.0

Set kubelet scraping to disabled

kubelet:
  enabled: false

Yet alertmanager is still firing alerts

KubeletDown (1 active)
--
alert: KubeletDown expr: absent(up{job="kubelet"}   == 1) for: 15m labels:   severity: critical annotations:   message: Kubelet has disappeared from Prometheus target discovery.   runbook_url: https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-kubeletdown                                                                                  Labels                   State                   Active Since                   Value                                                                                                                  alertname="KubeletDown"                                            severity="critical"                                                           firing                   2019-11-11 20:58:18.181974688 +0000 UTC                   1 | 

Is this expected?

Regards

vsliouniaev commented 4 years ago

Please upgrade to the latest chart version. I believe the alert has been fixed

acarsercan commented 4 years ago

Unfortunately no, I'm using the latest Chart 0.34.0 and kubelet alerts are still firing

vsliouniaev commented 4 years ago

Sounds like you're using the coreos chart that was removed from the coreos/prometheus-operator repository many months ago and is no longer supported.

acarsercan commented 4 years ago

Apologies, I'm not following. I'm using prometheus-operator-8.1.7 from

https://console.cloud.google.com/storage/browser/kubernetes-charts?prefix=prometheus-operator

vsliouniaev commented 4 years ago

Looks like these alerts are tied to

prometheusOperator:
  kubeletService:
    enabled: true
acarsercan commented 4 years ago

Apologies, that fixed it

vsliouniaev commented 4 years ago

This is definitely not clear from the chart. In reality both of these things need to be turned on/off for the components which monitor the kubelets to be enabled / disabled.