search

Yelp / sensu_handlers

Custom Sensu Handlers to support a multi-tenant environment, allowing checks themselves to emit the type of handler behavior they need in the event json
Apache License 2.0
75 stars 31 forks source link

exponential backoff sometimes crashes #2

Closed solarkennedy closed 10 years ago

solarkennedy commented 10 years ago

I think this might be happening in a log(0) event:

severities":["ok","warning","critical","unknown"],"name":"jira"},"output":"/etc/sensu/handlers/base.rb:173:in `log': Numerical result out of range - log (Errno::ERANGE)\n"}

I think this is the corresponding event data:

{"timestamp":"2014-09-04T00:37:24.241628-0700","level":"info","message":"handling event","event":{"client":{"name":"REDACTED","address":
"REDACTED","subscriptions":[REDACTED"],"safe_mode":true,"keepalive":{"handlers":["default"],"team":"operations","page":true,"realert
_every":"-1","runbook":"REDACTED},"timestamp":1409816239},"check":{"handlers":["default"],"command":" /usr/lib/nagios/plugins/check_http -H l
ocalhost -p 5052","subscribers":[],"standalone":true,"interval":60,"aggregate":false,"handle":true,"alert_after":300,"realert_every":"-1","runbook":"REDACTED","annotation":REDACTED","sla":"No SLA defined.","dependencies":[],"team":"REDACTED","irc_channels":"REDACTED","notification_email":"undef","ticket":false,"project":false,"page":true,"tip":false,"name":"check_marathon","i
ssued":1409816234,"executed":1409816234,"output":"CRITICAL - Socket timeout after 10 seconds\n","status":2,"duration":10.004,"history":["2","2","0","0","0"
,"0","0","0","2","2","0","0","0","0","0","0","2","2","2","2","2"]},"occurrences":5,"action":"create"},"handler":{"command":"/etc/sensu/handlers/jira.rb","t
ype":"pipe","severities":["ok","warning","critical","unknown"],"name":"jira"}}

I think I need to detect this case and.. do something?