sensu-plugins / sensu-plugin

A framework for writing Sensu plugins & handlers with Ruby.
http://sensuapp.org
MIT License
126 stars 117 forks source link

sensu-handler: filter_repeated never filter resolve events #142

Closed fessyfoo closed 7 years ago

fessyfoo commented 8 years ago

resolution events should not be filtered by filter_repeated

On-call personnel may not receive resolve notifications due to this filtering method. I believe this is a bad user experience which may lead some people to mistrust Sensu as a monitoring and alerting system. I think this is detrimental to our community -- cwjohnston

https://github.com/sensu-plugins/sensu-plugin/pull/136#issuecomment-232738538

note, this is not in opposition to the goal of moving filtering out of sensu-handler.rb #136 it just fixes an issue prior to moving filtering out as well.

fessyfoo commented 8 years ago

this PR is very similar to https://github.com/sensu-plugins/sensu-plugin/pull/103#issuecomment-158998616

and will suffer the same problem:

Even if your sequence of events didn't meet the Warning or Critical occurrence threshold necessary to invoke the handler, the handler will process the subsequent OK.

For example

0, 0, 0, 1, 0, 0, 0, 0 

May not trigger a notification because of the warning, but it'll always send the OK.

however taking that as requirement, then that's a very subtle thing and built into core filtering should consider it.

cwjohnston commented 7 years ago

I'm going to close this as the filter_repeated method is deprecated and I believe the occurrences extension provides an improved implementation using event watermarks.