We need a way to acknowledge specific alerts

[helgharbi]

We sometimes have expected alerts, but there is no appropriate way to acknowledge them. It would be interesting to have a charm action to ignore a specific alert pattern for a certain period of time. To avoid polluting the alert system with acknowledged alerts and continue to receive new alerts.

[gabrielcocenza]

I'm not sure if that responsibility should be on hardware-observer. Maybe worth a talk with the COS team to see if this can be done by Prometheus. We can reopen if we see that this should be done here