nerc-project / operations

Issues related to the operation of the NERC OpenShift environment
2 stars 0 forks source link

Remove CustomContainerMemoryUsage and CustomContainerCpuUsage Alerts in #alerts-nerc-ocp to Reduce Noise #730

Closed schwesig closed 2 months ago

schwesig commented 2 months ago

Remove CustomContainerMemoryUsage and CustomContainerCpuUsage Alerts in #alerts-nerc-ocp to Reduce Noise

Trigger

Discussion started in Slack with Justin, Chris, Thor https://massopencloud.slack.com/archives/C044ZCTUWCV/p1725628471733359

Is it easy to turn these alerts off?
We will never take action on CPU usage

Motivation

The current alerts for CustomContainerMemoryUsage and CustomContainerCpuUsage generate unnecessary noise in Alertmanager -> Slack Channel MOC #alerts-nerc-ocp. This issue aims to reduce alert overdose by removing these rules, as they have no follow-up actions in this channel (confirmed with Justin).

Completion Criteria

Description

The issue addresses the ongoing noise caused by two specific alert rules, which have been flagged as non-actionable. This follows the Slack discussion where it was agreed upon that these alerts should be removed.

Completion dates

Desired - 2024-09-13
Required - 2024-09-20

https://github.com/OCP-on-NERC/nerc-ocp-config/pull/532