search

iFixit / alertinator

Simplistic team-based alerting through email, sms and phone calls.
http://alertinator.readthedocs.org/
Other
9 stars 4 forks source link

Fixes #6: prevent multiples of the same alert #8

Closed entendu closed 6 years ago

entendu commented 8 years ago

A few of my assumptions based off #6:

Customizable intervals: Notify me when the check fails, when it's been failing for at least 30min, 6hours...

Since checks are (ostensibly) already on a timer, I think it makes more sense to notify after X failures/Y successes.

Be applied per (check-function, alert level) tuple

The way that check() is implemented, this is a problem. The alert level is based on which Exception class is used (e.g., AlertinatorWarningException), and in the case of success, the exception is never thrown. I have some ideas for implementing a better exception architecture (which would fit naturally into #3). As it stands, my code will alert on check-function only.

Resetting counters for a given check after a successful run (Or X successful runs)

Implemented

Storing any state locally on the file-system in whatever format makes sense.

Implemented as an Interface so someone could extend it to, say, mysql instead of the filesystem.

If $shouldAlert is true, the caller of this function will format a nice message using the $firstAlertTimestamp; something like "$alertMessage -- sustained for 30 minutes".

Somewhat annoying -- the message is defined when the Exception is thrown and as you know there is no $e->setMessage() method. I did some dirty business to help, but the messaging methods will have to be refactored to get the level of robustness you're looking for.

This PR should be fully backwards-compatible. Unit tests are included and docs are updated.

entendu commented 8 years ago

Merged #9 into here because I like green check marks.

djmetzle commented 7 years ago

CR 👍

lexahall commented 7 years ago

@dom do you have any QA tips for this?

djmetzle commented 7 years ago

@entendu this hasn't gotten touched for a year. dev_block :x: to stop polluting pulldasher.

entendu commented 7 years ago

You can read the readme and see how alertinator should now operate with this pull

n-fisher commented 6 years ago

un_dev_block 👍

I'm taking this over.

Will write up a QA instruction set!

n-fisher commented 6 years ago

Closing in favor of #11 (same commits, on a branch rather than entendu:master)