search

Icinga / icinga2

The core of our monitoring platform with a powerful configuration language and REST API.
https://icinga.com/docs/icinga2/latest
GNU General Public License v2.0
2k stars 574 forks source link

[dev.icinga.com #12004] Acknowledgement Notification Emails #4304

Closed icinga-migration closed 8 years ago

icinga-migration commented 8 years ago

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12004

Created by BongoEADGC6 on 2016-06-21 17:54:43 +00:00

Assignee: (none) Status: Rejected (closed on 2016-06-24 07:40:31 +00:00) Target Version: (none) Last Update: 2016-06-24 07:40:31 +00:00 (in Redmine)

Icinga Version: 2.4.10
Backport?: Not yet backported
Include in Changelog: 1

I have been seeing the following situation in my logs. When I acknowledge a service in Icinga-web The acknowledgement email gets sent to Slack, but not to the email notification.

Here are my configs:

template Notification "mail-service-notification-html" {
  if (service.vars.check_output_type == "Table") {
        command = "mail-service-notification-html-table"
  } else {
        command = "mail-service-notification-html"
  }

  //states = [ OK, Warning, Critical, Unknown ]
  //types = [ Problem, Acknowledgement, Recovery, Custom,
  //          FlappingStart, FlappingEnd,
  //          DowntimeStart, DowntimeEnd, DowntimeRemoved ]

  states = [ OK, Warning, Critical, Unknown ]
  types = [ Problem, Acknowledgement, Recovery, Custom,
            FlappingStart, FlappingEnd,
            DowntimeStart, DowntimeEnd, DowntimeRemoved ]

  // Set interval based on service variable if variable exists
  if (service.vars.mail_interval) {
      interval = service.vars.mail_interval
  } else if ("Storage" in service.groups || "ServerHardware"  in service.groups) {
      interval = 12h
  } else {
      interval = 24h
  }

  period = "24x7"
}

template Notification "slack-service-notification-py" {
  command = "slack-service-notification-py"

  states = [ OK, Warning, Critical, Unknown ]
  types = [ Problem, Acknowledgement, Recovery, Custom,
            FlappingStart, FlappingEnd,
            DowntimeStart, DowntimeEnd, DowntimeRemoved ]

  users = [ "slackuser" ]

  if (host.vars.mail_interval) {
      interval = host.vars.mail_interval
  } else if ("Storage" in host.groups || "ServerHardware"  in host.groups) {
      interval = 20m
  } else {
      interval = 12h
  }

  period = "24x7"
}

Here are the logs that are presented:

[2016-06-20 23:08:06 -0400] debug/Notification: Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:08:06 -0400] debug/Notification: State 'Critical', StateFilter OK, Warning, Unknown, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=15)
[2016-06-20 23:08:06 -0400] debug/Notification: User notification, Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:08:06 -0400] debug/Notification: User notification, State 'Critical', StateFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=-1)
[2016-06-20 23:08:06 -0400] debug/Notification: Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:08:06 -0400] debug/Notification: State 'Critical', StateFilter OK, Warning, Unknown, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=15)
[2016-06-20 23:08:06 -0400] debug/Notification: User notification, Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:08:06 -0400] notice/Notification: Not sending notifications for notification object 'NXXXXXXInfo!HP Hardware Health (ASM)!Notification HTML - NY - ServerHardware and user 'NY-ServerHardware-DG': type 'Acknowledgement' does not match type filter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd.
[2016-06-20 23:08:06 -0400] notice/ApiListener: Relaying 'event::SetAcknowledgement' message

[2016-06-20 23:13:45 -0400] debug/Notification: Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] debug/Notification: State 'Critical', StateFilter OK, Warning, Unknown, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=15)
[2016-06-20 23:13:45 -0400] debug/Notification: User notification, Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] debug/Notification: User notification, State 'Critical', StateFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=-1)
[2016-06-20 23:13:45 -0400] debug/Notification: Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] debug/Notification: State 'Critical', StateFilter OK, Warning, Unknown, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=15)
[2016-06-20 23:13:45 -0400] debug/Notification: User notification, Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] notice/Notification: Not sending notifications for notification object 'NCCCCCC04!HP Hardware Health (ASM)!Notification HTML - NY - ServerHardware and user 'NY-ServerHardware-DG': type 'Acknowledgement' does not match type filter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd.
icinga-migration commented 8 years ago

Updated by BongoEADGC6 on 2016-06-21 18:11:25 +00:00

This is the full message for an affected notification

[2016-06-20 23:13:45 -0400] notice/Comment: Added comment 'NCCCCCC04!HP Hardware Health (ASM)!MDBAL01ICINGA2.radnet.rdlx.pvt-1466478825-0'.
[2016-06-20 23:13:45 -0400] information/Checkable: Checking for configured notifications for object 'NCCCCCC04!HP Hardware Health (ASM)'
[2016-06-20 23:13:45 -0400] debug/Checkable: Checkable 'NCCCCCC04!HP Hardware Health (ASM)' has 2 notification(s).
[2016-06-20 23:13:45 -0400] notice/Notification: Attempting to send notifications for notification object 'NCCCCCC04!HP Hardware Health (ASM)!Slack Service - NY - ServerHardwa
re'.
[2016-06-20 23:13:45 -0400] debug/Notification: Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem,
Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] debug/Notification: State 'Critical', StateFilter OK, Warning, Unknown, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, Flapp
ingStart and FlappingEnd (FState=4, StateFilter=15)
[2016-06-20 23:13:45 -0400] debug/Notification: User notification, Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowl
edgement, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] debug/Notification: User notification, State 'Critical', StateFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgem
ent, Problem, Recovery, FlappingStart and FlappingEnd (FState=4, StateFilter=-1)
[2016-06-20 23:13:45 -0400] information/Notification: Sending notification 'NCCCCCC04!HP Hardware Health (ASM)!Slack Service - NY - ServerHardware' for user 'slackuser'
[2016-06-20 23:13:45 -0400] debug/DbEvents: add notification history for 'NCCCCCC04!HP Hardware Health (ASM)'
[2016-06-20 23:13:45 -0400] debug/DbEvents: add contact notification history for service 'NCCCCCC04!HP Hardware Health (ASM)' and user 'slackuser'.
[2016-06-20 23:13:45 -0400] notice/Notification: Attempting to send notifications for notification object 'NCCCCCC04!HP Hardware Health (ASM)!Notification HTML - NY - ServerHa
rdware'.
[2016-06-20 23:13:45 -0400] debug/Notification: Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Up, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem,
Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] debug/Notification: State 'Critical', StateFilter OK, Warning, Unknown, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, Flapp
ingStart and FlappingEnd (FState=4, StateFilter=15)
[2016-06-20 23:13:45 -0400] debug/Notification: User notification, Type 'Acknowledgement', TypeFilter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledge
ment, Problem, Recovery, FlappingStart and FlappingEnd (FType=16, TypeFilter=511)
[2016-06-20 23:13:45 -0400] notice/Notification: Not sending notifications for notification object 'NCCCCCC04!HP Hardware Health (ASM)!Notification HTML - NY - ServerHardware and user 'NY-ServerHardware-DG': type 'Acknowledgement' does not match type filter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd.
[2016-06-20 23:13:45 -0400] notice/Notification: Notification filters for user 'NY-ServerHardware-DG' not matched. Not sending notification.
[2016-06-20 23:13:45 -0400] debug/DbEvents: add notification history for 'NCCCCCC04!HP Hardware Health (ASM)'
[2016-06-20 23:13:45 -0400] debug/DbEvents: add acknowledgement for 'NCCCCCC04!HP Hardware Health (ASM)'
[2016-06-20 23:13:45 -0400] notice/ApiListener: Relaying 'event::SetAcknowledgement' message
[2016-06-20 23:13:45 -0400] debug/DbEvents: add acknowledgement history for 'NCCCCCC04!HP Hardware Health (ASM)'
icinga-migration commented 8 years ago

Updated by gbeutner on 2016-06-22 07:23:10 +00:00

@dnsmichi: Is that the problem we talked about yesterday?

icinga-migration commented 8 years ago

Updated by mfriedrich on 2016-06-22 12:24:06 +00:00

The log message is wrong, since NotificationFilterToString() does not select between types/states. That part will be fixed soon.

A correct log message (from #11445 branch) would look like this:

types = [ Problem, Recovery ]

[2016-06-22 14:14:51 +0200] notice/Notification: Attempting to send notifications for notification object '11445-host!11445-host-notification'.
[2016-06-22 14:14:51 +0200] debug/Notification: Type 'Acknowledgement', TypeFilter Problem and Recovery (FType=16, TypeFilter=96)
[2016-06-22 14:14:51 +0200] notice/Notification: Not sending notifications for notification object '11445-host!11445-host-notification': type 'Acknowledgement' does not match type filter Problem and Recovery.

The provided configuration is incomplete - there are at least the following configuration bits missing:

Best would be to provide these fully compiled objects from either the cli command 'object list' or directly from the API. That will help to understand and reproduce the issue you're having.

icinga-migration commented 8 years ago

Updated by BongoEADGC6 on 2016-06-22 15:52:26 +00:00

Object 'NJNERC1UICW!HP Agent Status!Notification HTML - NY - ServerHardware' of type 'Notification':
  % declared in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 241:1-241:71
  * __name = "NJNERC1UICW!HP Agent Status!Notification HTML - NY - ServerHardware"
  * command = "mail-service-notification-html"
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-notifications.conf', lines 43:2-43:43
  * command_endpoint = ""
  * host_name = "NJNERC1UICW"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 241:1-241:71
  * interval = 43200
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-notifications.conf', lines 60:7-60:20
  * name = "Notification HTML - NY - ServerHardware"
  * package = "_etc"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 241:1-241:71
  * period = "24x7"
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-notifications.conf', lines 65:3-65:17
  * service_name = "HP Agent Status"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 241:1-241:71
  * states = [ 1, 2, 4, 8 ]
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-notifications.conf', lines 51:3-51:45
  * templates = [ "Notification HTML - NY - ServerHardware", "mail-service-notification-html" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 241:1-241:71
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-notifications.conf', lines 39:1-39:54
  * times = null
  * type = "Notification"
  * types = [ 32, 16, 64, 8, 128, 256, 1, 2, 4 ]
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-notifications.conf', lines 52:3-54:57
  * user_groups = [ "NY-ServerHardware" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 244:4-244:40
  * users = null
  * vars
    * category = "Server Hardware"
      % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 246:4-246:36
  * zone = "ny-satellite"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/notifications.conf', lines 241:1-241:71

There is a example from the object list

icinga-migration commented 8 years ago

Updated by mfriedrich on 2016-06-23 12:13:15 +00:00

It would be helpful to see exactly the notification and user objects from your logs.

The User object type filter is preventing the notification being sent.

[2016-06-20 23:13:45 -0400] notice/Notification: Not sending notifications for notification object 'NCCCCCC04!HP Hardware Health (ASM)!Notification HTML - NY - ServerHardware and user 'NY-ServerHardware-DG': type 'Acknowledgement' does not match type filter OK, Warning, Unknown, Down, DowntimeEnd, DowntimeRemoved, Custom, Acknowledgement, Problem, Recovery, FlappingStart and FlappingEnd.

Using the current snapshot packages, I'm not able to reproduce the issue.

object Host "11445-host" {
  check_command = "dummy"
  max_check_attempts = 1
  check_interval = 5m
  retry_interval = 4m
}

object User "11445-user" {
  states = [ Up, Down, Critical, OK ]
  types = [ Problem, Recovery, Acknowledgement ]
}

apply Notification "11445-host-notification" to Host {
  command = "mail-host-notification"

  users = [ "11445-user" ]

  states = [ Up, Down ]
  //types = [ "Problem", "Recovery", "Acknowledgement" ]
  types = [ Problem, Recovery, Acknowledgement ]
  assign where host.name == "11445-host"
}
~

[2016-06-23 14:11:11 +0200] information/ExternalCommandListener: Executing external command: [1466683871] ACKNOWLEDGE_HOST_PROBLEM;11445-host;2;1;0;icingaadmin;fsfsfdsfsd
[2016-06-23 14:11:11 +0200] notice/ExternalCommandProcessor: Setting acknowledgement for host '11445-host'
[2016-06-23 14:11:11 +0200] information/ConfigCompiler: Compiling config file: /usr/local/icinga2/var/lib/icinga2/api/packages/_api/mbmif.int.netways.de-1442309540-1/conf.d/comments/11445-host!mbmif.int.netways.de-1466683871-0.conf
[2016-06-23 14:11:11 +0200] debug/ConfigItem: Activating object '11445-host!mbmif.int.netways.de-1466683871-0' of type 'Comment'
[2016-06-23 14:11:11 +0200] debug/ApiListener: Sent update for object '11445-host!mbmif.int.netways.de-1466683871-0': {"config":"object Comment \"mbmif.int.netways.de-1466683871-0\" ignore_on_error {\n\tauthor = \"icingaadmin\"\n\tentry_time = 1466683871.392597\n\tentry_type = 4.000000\n\texpire_time = 0.000000\n\thost_name = \"11445-host\"\n\ttext = \"fsfsfdsfsd\"\n\tversion = 1466683871.393262\n}\n","modified_attributes":{},"name":"11445-host!mbmif.int.netways.de-1466683871-0","original_attributes":[],"type":"Comment","version":1466683871.3932619095}
[2016-06-23 14:11:11 +0200] notice/Comment: Added comment '11445-host!mbmif.int.netways.de-1466683871-0'.
[2016-06-23 14:11:11 +0200] information/Checkable: Checking for configured notifications for object '11445-host'
[2016-06-23 14:11:11 +0200] debug/Checkable: Checkable '11445-host' has 1 notification(s).
[2016-06-23 14:11:11 +0200] notice/Notification: Attempting to send notifications for notification object '11445-host!11445-host-notification'.
[2016-06-23 14:11:11 +0200] information/Notification: Sending notification '11445-host!11445-host-notification' for user '11445-user'
[2016-06-23 14:11:11 +0200] debug/DbEvents: add notification history for '11445-host'
[2016-06-23 14:11:11 +0200] debug/DbEvents: add contact notification history for service '11445-host' and user '11445-user'.
[2016-06-23 14:11:11 +0200] debug/DbEvents: add log entry history for '11445-host'
[2016-06-23 14:11:11 +0200] information/Notification: Completed sending notification '11445-host!11445-host-notification' for checkable '11445-host' and user '11445-user'.

From a fixed branch in 1445, this looks the same (tests are a final requirement for merge):

object Host "11445-host" {
  check_command = "dummy"
}

object User "11445-user" {
  states = [ Up, Down, Critical, OK ]
  types = [ Problem, Recovery, Acknowledgement ]
}

apply Notification "11445-host-notification" to Host {
  command = "mail-host-notification"

  users = [ "11445-user" ]

  states = [ Up, Down ]
  types = [ "Problem", "Recovery", "Acknowledgement" ]
  assign where host.name == "11445-host"
}

[2016-06-23 14:01:15 +0200] information/ExternalCommandListener: Executing external command: [1466683275] ACKNOWLEDGE_HOST_PROBLEM;11445-host;2;1;0;icingaadmin;dfdfsdfsdfsdfsdfs
[2016-06-23 14:01:15 +0200] notice/ExternalCommandProcessor: Setting acknowledgement for host '11445-host'
[2016-06-23 14:01:15 +0200] information/ConfigCompiler: Compiling config file: /usr/local/icinga2/var/lib/icinga2/api/packages/_api/mbmif.int.netways.de-1442309540-1/conf.d/comments/11445-host!mbmif.int.netways.de-1466683275-0.conf
[2016-06-23 14:01:15 +0200] debug/ConfigItem: Activating object '11445-host!mbmif.int.netways.de-1466683275-0' of type 'Comment'
[2016-06-23 14:01:15 +0200] debug/ApiListener: Sent update for object '11445-host!mbmif.int.netways.de-1466683275-0': {"config":"object Comment \"mbmif.int.netways.de-1466683275-0\" ignore_on_error {\n\tauthor = \"icingaadmin\"\n\tentry_time = 1466683275.470960\n\tentry_type = 4.000000\n\texpire_time = 0.000000\n\thost_name = \"11445-host\"\n\ttext = \"dfdfsdfsdfsdfsdfs\"\n\tversion = 1466683275.471027\n}\n","modified_attributes":{},"name":"11445-host!mbmif.int.netways.de-1466683275-0","original_attributes":[],"type":"Comment","version":1466683275.4710268974}
[2016-06-23 14:01:15 +0200] notice/Comment: Added comment '11445-host!mbmif.int.netways.de-1466683275-0'.
[2016-06-23 14:01:15 +0200] information/Checkable: Checking for configured notifications for object '11445-host'
[2016-06-23 14:01:15 +0200] debug/Checkable: Checkable '11445-host' has 1 notification(s).
[2016-06-23 14:01:15 +0200] notice/Notification: Attempting to send notifications for notification object '11445-host!11445-host-notification'.
[2016-06-23 14:01:15 +0200] information/Notification: Sending notification '11445-host!11445-host-notification' for user '11445-user'
[2016-06-23 14:01:15 +0200] debug/DbEvents: add notification history for '11445-host'
[2016-06-23 14:01:15 +0200] debug/DbEvents: add contact notification history for service '11445-host' and user '11445-user'.
[2016-06-23 14:01:15 +0200] debug/DbEvents: add acknowledgement for '11445-host'
[2016-06-23 14:01:15 +0200] debug/DbEvents: add acknowledgement history for '11445-host'
[2016-06-23 14:01:15 +0200] debug/DbEvents: add log entry history for '11445-host'
[2016-06-23 14:01:15 +0200] information/Notification: Completed sending notification '11445-host!11445-host-notification' for checkable '11445-host' and user '11445-user'.
icinga-migration commented 8 years ago

Updated by BongoEADGC6 on 2016-06-23 14:35:09 +00:00

This is one of the user configs that I have that isn't working

Object 'NY-ServerHardware-DG' of type 'User':
  % declared in '/etc/icinga2/zones.d/global-templates/users.conf', lines 452:1-452:34
  * __name = "NY-ServerHardware-DG"
  * display_name = "NY ServerHardware DG"
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 454:4-454:40
  * email = "IcingaNewYorkServerHardware@*****.COM"
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 455:4-455:51
  * enable_notifications = true
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 33:5-33:31
  * groups = [ "NY-ServerHardware" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 456:4-456:35
  * name = "NY-ServerHardware-DG"
  * package = "_etc"
  * pager = ""
  * period = ""
  * states = [ 1, 4, 8, 2, 16, 32 ]
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 31:5-31:57
  * templates = [ "NY-ServerHardware-DG", "generic-contact" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 452:1-452:34
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 30:1-30:31
  * type = "User"
  * types = [ 32, 8, 128, 256, 64, 1, 2, 4 ]
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 32:5-32:114
  * vars = null
  * zone = "global-templates"

And this is the slack user that does work.

Object 'slackuser' of type 'User':
  % declared in '/etc/icinga2/zones.d/global-templates/users.conf', lines 15:1-15:23
  * __name = "slackuser"
  * display_name = "Slack Dummy Account"
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 18:3-18:38
  * email = "slack@localhost"
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 20:3-20:27
  * enable_notifications = true
  * groups = [ ]
  * name = "slackuser"
  * package = "_etc"
  * pager = ""
  * period = ""
  * states = null
  * templates = [ "slackuser", "generic-user" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/users.conf', lines 15:1-15:23
    % = modified in '/etc/icinga2/zones.d/global-templates/templates/templates-services.conf', lines 66:1-66:28
  * type = "User"
  * types = null
  * vars = null
  * zone = "global-templates"

I just realized that they are using two different templates. One that we had migrated from Icinga1.

template User "generic-user" {

}

template User "generic-contact" {
    states = [ OK, Critical, Unknown, Warning, Up, Down ]
    types = [ Problem, Custom,
              FlappingStart, FlappingEnd, Recovery,
              DowntimeStart, DowntimeEnd, DowntimeRemoved ]
    enable_notifications = true

}

I went and commented out the settings in "generic-contact."

This is more than likely the reason why we've been seeing this issue. I will test and confirm.

icinga-migration commented 8 years ago

Updated by BongoEADGC6 on 2016-06-23 19:31:52 +00:00

Looks like that was the case. Sorry to raise any alarms.

This non-bug can be closed due to user error.

icinga-migration commented 8 years ago

Updated by mfriedrich on 2016-06-24 07:40:31 +00:00

No worries. It was a good kick-off into fixing the wrong log lines with #11445 and properly testing the fix while trying to reproduce your issue :)