search

kubernetes-monitoring / kubernetes-mixin

A set of Grafana dashboards and Prometheus alerts for Kubernetes.
Apache License 2.0
2.11k stars 598 forks source link

CPU Throttling trigger should be Way higher! #515

Open dioguerra opened 3 years ago

dioguerra commented 3 years ago

Throttling on pods is set to trigger alarm when it is above 25% This should be much much higher because if the applications are well set up they would consumer efficiently the resources.

I would say this Throttling should be only triggered for CPU usage above 95% to 90%.

It's normal for my clusters to always trigger an alarm for my prometheus-node-exporter pods as they always have a a CPU consumption that is around the 90% requested CPU

metalmatze commented 3 years ago

You can configure this in your local configuration.

Here's the value to overwrite: https://github.com/kubernetes-monitoring/kubernetes-mixin/blob/master/alerts/resource_alerts.libsonnet#L12 It can be placed added in the top level config+:: to overwrite the default values. That's why we chose jsonnet after all, cause it's easier to individualize.