search

fluxcd / notification-controller

The GitOps Toolkit event forwarder and notification dispatcher
https://fluxcd.io
Apache License 2.0
150 stars 132 forks source link

Document exclusionList examples to ignore commonly spammy notifications #283

Open kingdonb opened 2 years ago

kingdonb commented 2 years ago

This doc has a section for examples related to Notification Controller:

https://github.com/fluxcd/notification-controller/blob/main/docs/spec/v1beta1/alert.md

It shows up in the published docs here:

https://fluxcd.io/docs/components/notification/alert/#example

We thought of this during Bug Scrub yesterday, the suggestion was to provide some (tested) examples for the exclusionList of notification rules that can be safely ignored, or are frequently ignored, with regex'es for the more commonly requested ones (so people do not have to figure this out on their own, or guess at what regex will match)

Some examples are:

^Dependencies do not meet ready condition, retrying in.*
^unable to clone '.*': dial tcp: .*: server misbehaving$
^HelmChart '.*' is not ready$

By providing more guidance around this including valid regexp examples that actually match the messages that are emitted, we'll make it easier for people to construct good alerting rules on the first or second try.

These things will likely be easier to address and completely document after the Status Conditions updates have all landed. We can't assume that will solve all notification issues though, it would be good to have a bank of regex exclude examples.

A few separate ideas that we considered to be out of scope, or that belong somewhere else were part of this discussion, I'll be opening separate issues for those and link back here.

*Writing tests to make sure the examples in these docs still match the messages emitted seems like it would be an important part of such a doc.

kingdonb commented 2 years ago

This was the discussion thread that sparked our interest in this issue:

https://github.com/fluxcd/flux2/discussions/1868