nisabek / icinga2-slack-notifications

Icinga2 notification integration with slack
Apache License 2.0
71 stars 36 forks source link

Error when host.output is empty (no notification sent) #23

Closed winem closed 5 years ago

winem commented 5 years ago

The slack-notitfications-command.conf fails with this error:

Location: in /var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf: 80:29-80:75
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(78): 
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(79):     log(LogDebug, "slack-notifications", "Sending notification...generating notification text")
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(80):     var short_host_output = host_output.substr(0, plugin_output_max_length)
                                                                                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(81):     var host_name_with_link = "<" + icinga2_base_url + "/monitoring/host/show?host=" + host_name + "|" + host_display_name + ">"
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(82):     var text = "error crafting payload"

[2019-02-06 16:14:47 +0000] warning/PluginNotificationTask: Notification command for object 'SCHR-pbr8dns0!load1' (PID: -1, arguments: 'sh' '-c' '') terminated with exit code 3, output: Error: Argument is not a callable object.
Location: in /var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf: 80:29-80:75
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(78): 
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(79):     log(LogDebug, "slack-notifications", "Sending notification...generating notification text")
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(80):     var short_host_output = host_output.substr(0, plugin_output_max_length)
                                                                                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(81):     var host_name_with_link = "<" + icinga2_base_url + "/monitoring/host/show?host=" + host_name + "|" + host_display_name + ">"
/var/lib/icinga2/api/zones/global-templates/_etc/slack-notifications-command.conf(82): var text = "error crafting payload"

After some testing I found out that it's just related to "virtual" hosts without an icinga2 agent installed and found a repro:

  1. Create a host like with check_command dummy. Example:
    address = "127.0.0.1"
    check_command = "dummy"
    max_check_attempts = "5"
    check_interval = 1m
    retry_interval = 30s
    enable_notifications = true
    enable_active_checks = false
    enable_passive_checks = true
    enable_event_handler = false
    enable_flapping = false
    enable_perfdata = true
    volatile = false
    zone = "yblf-om"
    vars.slack_notifications = "enabled"
    vars.sms_enabled = true
    vars.swap_available = false
}

Now trigger a host or service notification for this host (custom notifications are fine to repro it, too). You should see the error from above.

To fix it, I just use Process check result to set the host state to UP for example and pass some output to the check result. Now, notifications work.

I think we need a check if $host.output$ is available before we refer to it and have a sane default or fallback.

Setup information:

icinga2 - The Icinga 2 network monitoring daemon (version: r2.10.2-1)

Copyright (c) 2012-2018 Icinga Development Team (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Ubuntu
  Platform version: 18.04.1 LTS (Bionic Beaver)
  Kernel: Linux
  Kernel version: 4.15.0-42-generic
  Architecture: x86_64

Build information:
  Compiler: GNU 7.3.0
  Build host: 8baa592f9b22

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid
winem commented 5 years ago

Will provide a PR with a fix later today.