warning: event filtering in sensu-plugin is deprecated

[bwilder]

Using sensu v.0.26.5 and sensu-plugins-mailer v.1.0.0. My mailer config is located in /etc/sensu/config.json (sensitive info changed):

{
  "handlers": {
  ...
    "mail": {
      "type": "pipe",
      "command": "handler-mailer.rb -j mail"
    },
  ...
  "mail": {
    "admin_gui": "http://127.0.0.1:3000/",
    "mail_from": "noreply@mydomain.com",
    "mail_to": "me@mydomain.com",
    "smtp_address": "localhost",
    "smtp_port": "25",
    "smtp_domain": "mydomain.com"
  }
}

I'm receiving emails, but the logs are getting flooded with the following:

sensu-server.log:

{"timestamp":"2016-11-02T15:21:55.183214+0000","level":"info","message":"handler output","handler":{"type":"pipe","command":"handler-mailer.rb -j mail","name":"mail"},"output":["warning: event filtering in sensu-plugin is deprecated, see http://bit.ly/sensu-plugin\nwarning: occurrence filtering in sensu-plugin is deprecated, see http://bit.ly/sensu-plugin\nonly handling every 60 occurrences: bw-sensu-server_i-d1954ae1/test-mail-handler\n"]}

sensu-client.log:

{"timestamp":"2016-11-02T15:21:54.677492+0000","level":"info","message":"publishing check result","payload":{"client":"bw-sensu-server_i-d1954ae1","check":{"type":"metric","command":"return 2","handlers":["mail"],"standalone":true,"interval":30,"name":"test-mail-handler","issued":1478100114,"executed":1478100114,"duration":0.001,"output":"","status":2}}}

The workaround is to add "enable_deprecated_filtering": false to the check that uses the handler. In a small sensu implementation, this is inconvenient. In a large implementation w/lots of distributed hosts not under your control, and dozens of checks per host, this is a significant burden.

Is there a timeframe for a fix?

[Evesy]

As of sensu-plugin version 1.4.5 deprecated filtering can be disabled globally via the sensu_plugin configuration scope, so there's no longer the annoyance of having to specify this in every check i.e:

"sensu_plugin": {
  "disable_deprecated_filtering": true
}

Deprecated filtering is disabled by default in sensu-plugin 2.0.0, however the gem version constraints on this handler plugin would need to be updated.

@majormoses Any thoughts on how to approach moving handler plugins over to more recent versions of sensu-plugin? Gem dependency could be changed from ~> 1.2 to > 1.2 however this would be a potentially breaking change for a lot of people.

[majormoses]

Definitely a breaking change and don't have a quick solution that I like due to sensu-plugin 2.x requires ruby 2.4 which breaks a lot of sensu installs and plugins (some with no path forward currently until upstream issues are resolved). A hacky solution would be to accomplish this through a ruby extension to conditionally install a newer version at install time vs build time based on ruby version like I did here: https://github.com/sensu-plugins/sensu-plugins-supervisor/pull/22. I will say that that was a terrible work around that should have never been required and makes perfect sense given our options. I do not

I will write up an issue to track this for all plugins and will link back here when I do.

[majormoses]

As promised I have created this uber issue: https://github.com/sensu-plugins/sensu-plugins-feature-requests/issues/26

[majormoses]

@Evesy more note is that you can always install sens-plugins 1.4 outside of the context of sensu dependencies and get the desired behavior without making it a hard dep that needs to be dealt with more carefully as 1.4 will satisfy the dependency of ~> 1.2

[majormoses]

Gem dependency could be changed from ~> 1.2 to > 1.2

Is basically almost the same thing other than it locks it to the major and allow 1.2 to satisfy the dependency. If anything if we did a release to bump this prior to 2.0 we would still need to lock it at the major so it would be ~> 1.4.5

[arthurzenika]

Am bumping into this warning, but as an error...

{"timestamp":"2017-08-31T10:49:09.401610+0000","level":"error","message":"handler output","handler":{"command":"/usr/local/bin/handler-mailer.rb","type":"pipe","name":"mailer"},"event":{"id":"4da25df0-89e5-481b-922a-665226d878f6"},"output":["/var/lib/gems/2.1.0/gems/sensu-plugins-mailer-1.2.0/bin/handler-mailer.rb:175:in `build_body': undefined method `[]' for nil:NilClass (NoMethodError)\n\tfrom /var/lib/gems/2.1.0/gems/sensu-plugins-mailer-1.2.0/bin/handler-mailer.rb:202:in `handle'\n\tfrom /var/lib/gems/2.1.0/gems/sensu-plugin-1.4.5/lib/sensu-handler.rb:99:in `block in <class:Handler>'\nwarning: event filtering in sensu-plugin is deprecated, see http://bit.ly/sensu-plugin\nwarning: occurrence filtering in sensu-plugin is deprecated, see http://bit.ly/sensu-plugin\n"]}

I have tried to insert the disable flag with not much success...

"sensu_plugin": {
  "disable_deprecated_filtering": true
}

(with true and false, I find the flag a bit confusing, and unfortunately undocumented).

Is this another error ? another issue ? How can I debug this ?

I must admit reading through https://blog.sensuapp.org/deprecating-event-filtering-in-sensu-plugin-b60c7c500be3 as a new adopter and tester of sensu is not a great experience... especially when trying to make my first sensu setup simply send out some emails (and seeing that the enterprise version does that out of the box). I know (and work in) open source, so I understand that this is on a best effort basis, but I wanted to reach out with that feeling and experience, I think you might be loosing new users over this kind of complication.

I hope to continue working on sensu...

[GhostLyrics]

@arthurlogilab

• Have you added occurrences and refresh attributes to your check definition?
• Have you checked if the Ruby embedded into Sensu has the sensu-extensions-occurrences' gem installed?

Example output from an Ubuntu machine:

sudo /opt/sensu/embedded/bin/gem list|grep occ
sensu-extensions-occurrences (1.1.0)

[arthurzenika]

# /opt/sensu/embedded/bin/gem list| grep occ
sensu-extensions-occurrences (1.2.0)

am a bit worried that am mixing gems here though

#/opt/sensu/embedded/bin/gem list| grep plugin
sensu-plugin (2.0.1, 1.4.5)
#gem list | grep sensu
sensu-plugin (1.4.5)
sensu-plugins-mailer (1.2.0)
/opt/sensu/embedded/bin/gem list| grep mailer
[nothing]

am deploying sensu with https://github.com/saltstack-formulas/sensu-formula/ and this makes me think that I might have to set the embedded_ruby flag https://github.com/saltstack-formulas/sensu-formula/blob/master/pillar.example#L13

thanks for your help!

[arthurzenika]

oh, and refresh was set, have added occurrences:1 but that doesn't seem to solve it.

[GhostLyrics]

@arthurlogilab Does your handler include a section like the following?

{
  "handlers": {
    "mail": {
      "command": "handler-mailer.rb -t /etc/sensu/handlers/mail_template.erb -j mail",
      "type": "pipe",
      "filters": [
        "occurences"
      ],
      "severities": [
        "critical"
      ]
    }
  },
  "mail": {
    "mail_from": "REDACTED@REDACTED",
    "mail_to": "root"
  }
}

[arthurzenika]

ok, it seems that it was simply the missing -j mail that was a problem.

I've done a pull request to update the README https://github.com/sensu-plugins/sensu-plugins-mailer/pull/56

Maybe the error message could be a little bit more explicit, to avoid people chasing down the deprecation warning.

Is the mailer key the default key used or is has to be explicitly given like in your example ?

[GhostLyrics]

AFAIR mailer is the default key.

[majormoses]

I am going to close this out as we enhanced the docs to help users avoid/troubleshoot the situation.