search

bcgov / DITP-DevOps

Digital Identity and Trust Program Team's DevOps Documentation Repository
Apache License 2.0
2 stars 5 forks source link

Add alerts for pods that are being "heavily" throttled #183

Closed WadeBarnes closed 2 months ago

WadeBarnes commented 3 months ago

Our new monitoring stack provides far better insight into the our system metrics than any of the other tools available for the platform. Specifically, we now have in depth insight in to the throttling metrics of our pods.

Please setup alerts to trigger notifications when pods are throttled >25% for more than 5 minutes.

This should be done after the review of the related Throttling Tickets.

i5okie commented 2 months ago

Alerts have been created in Grafana with notifications being sent to RocketChat.