sensu / sensu-go

Simple. Scalable. Multi-cloud monitoring.
https://sensu.io
MIT License
1.02k stars 175 forks source link

Delete silenced entries when "expire on resolve" is set to true #664

Closed mercul3s closed 6 years ago

mercul3s commented 6 years ago

We didn't have a clear definition or mechanism for resolving events when the silenced api was written. When https://github.com/sensu/sensu-go/pull/658 and https://github.com/sensu/sensu-go/pull/652 are merged, we will have event resolution. When events are resolved, we'll need to clean up any silenced entries for that event's check/subscription that have ExpireOnResolve set to true.

grepory commented 6 years ago

This needs a test case / acceptance test of some kind.

grepory commented 6 years ago

This involves silencing a failing check, force resolution, ensure that the silence is removed and event no longer silenced after resolution.

amdprophet commented 6 years ago

This requires event actions (create, flapping, resolve) to be in place before it can be implemented. I could swear that we had an issue to implement event actions but I'm not having any luck finding it.

palourde commented 6 years ago

@amdprophet The states are determined here but we don't have a resolve one: https://github.com/sensu/sensu-go/blob/f4509a89cbbc2db6b7bcf05a810df7681b94a706/backend/eventd/flapping.go#L23-L31

That being said, as mentioned in https://github.com/sensu/sensu-go/issues/587#issuecomment-348967972, I don't think we can rely on the state to determine if an event indicates a resolution, but we can probably rely on this logic instead https://github.com/sensu/sensu-go/blob/f4509a89cbbc2db6b7bcf05a810df7681b94a706/backend/pipelined/filter.go#L138-L147

Unless we add a state like resolving?

portertech commented 6 years ago

I like the idea of using history.

resolving is a transition, no? I don't like the idea of using event.Check.State for that.

amdprophet commented 6 years ago

isResolution looks like what I should be using. I was expecting to see something similar to what we had in 1.x. Thanks!