email notifier: specifing different receivers for different consul service

[imrangit]

Is there a way to specify different receivers for different consul service to notify?

[claudio-viola]

up, wondering this as well

[imrangit]

we have tried setting up custom profiles for different services/events.. but we are not able to get this to work? there isnt enough documentation. We even tried upgrading consul to the latest.

If the feature exist, can someone give a pointer? if not then, can we add this feature?

[Gerrrr]

Hello,

In our project we really miss this feature. Here is a proposal on how it can be done.

Named notifiers of a specific type

Currently, it is only possible to have 1 configuration for 1 notifier type, e.g. consul-alerts/config/notifiers/email.

With multiple configurations, it will be possible to send emails to different receivers or post to different slack channels. API example:

• consul-alerts/config/notifiers/emails/<email-notifier-name> - a place to keep named notifiers of type emails
• consul-alerts/config/notifiers/slacks/<slack-notifier-name> - a place to keep named notifiers of type slack

Then, in the notification profiles it should be possible not only to turn on/off the notifier, but also specify a notifier name:

{
  "Interval": 10,
  "NotifList": {
    "log":false,
    "emails": ["team2", "team-3"]
  }
}

Notification Profile Activation using regular expressions

Currently, for every service, check or host to be notified about, there should be a record in consul-alerts/config/notif-selection/[services|nodes|checks]/ directories. However, It is not possible to configure not just one service, but a group of services. To solve this, we can use consul-alerts/config/notif-selection/[services|nodes|checks] keys for the json, mapping groups of services (matched by regexp) to a list of notification profiles. Example:

Key: consul-alerts/config/notif-selection/services Value:

{
  "^team-1-.*$": ["profile-1", "profile-2"]
}

[Gerrrr]

Hey guys, @imrangit @claudio-viola @fusiondog would you mind sharing your feedback on the proposal? If you are fine with it, I'll implement it over this week.

[imrangit]

I have gone the custom notifier route. If you want, I can do the PR.

-Imran

[fusiondog]

There are some notes from a similar ticket here: https://github.com/AcalephStorage/consul-alerts/issues/118

[fusiondog]

Basically the important note from that is the idea of using variable overrides in the profile. Possibly in ways similar to your proposal. Like having the config overrides as the key value instead of just true. I would prefer it to be a general solution and not specific to email or slack. Just send the overrides to whichever notifier is triggered.

I also do like the idea of matching notification on regular expressions.

[Gerrrr]

Hi @fusiondog Thanks for your feedback. I really like your suggestion about variable overrides in the profiles instead of adding emails, slacks, etc. I am currently working on the implementation of the overrides and I noticed that the information about the builtin notifiers is defined 2 times - in the consul configuration and also in the actual notifiers.

It will make the implementation of the variable overrides much simpler if I merge the rest of the configuration information into the instances defined in the notifier package, i.e. during the configuration load it will be loaded directly to the corresponding instance. Would you mind if I do so? Is there a reason to have the notifier instances decoupled from the configuration?

[fusiondog]

Do you mean the config here: https://github.com/AcalephStorage/consul-alerts/blob/master/consul-alerts.go#L239

And here, for instance: https://github.com/AcalephStorage/consul-alerts/blob/master/notifier/email-notifier.go#L16

?

If so I think it is primarily to feed into this list of notifiers for processing: https://github.com/AcalephStorage/consul-alerts/blob/e008af7894814c949f40c861fc7add881fedba94/send-notifs.go#L50

It may also be because they are in separate packages, but I didn't originally write that part so I'm not positive.
Maybe you could find a way to get the notifiers to register themselves into a shared array. That may make it easier to add future builtin notifiers. I have no problem with it being refactored if it makes it easier and works the same.

[ThePletch]

Any updates on this? My team is waiting on this feature before we add this package to our system.

[Gerrrr]

Hi @ThePletch The PR is available and we are using it in our fork since early January. The Travis tests are failing because I did not figure how to make new vendored dependencies available in CI (since they are not yet in AcalephStorage/consul-alerts/Godeps, I can't go get them).

If you are going to try it, please let me know if you'd have any feedback, I am happy to fix problems you find or extend if you miss anything.

[moadqassem]

Did anyone try to override the slack variables? I have a slack notifier with the consul-alerts/config/notifiers/slack/channel , consul-alerts/config/notifiers/slack/enabled , consul-alerts/config/notifiers/slack/detailed , consul-alerts/config/notifiers/slack/url and I have a different profile called override-slack, {"Interval":1, "NotifList":{ "slack":true }, "VarOverrides": { "slack": { "channel": "#xxx", "enabled" : true, "detailed": true, "url": "xxx"}}}

And I am selecting the override profile: consul-alerts/config/notif-selection/services {"service.*$": "override-slack"}

However it doesn't work and the vars are not being used. I thought this could be related to this field ignoring:

https://github.com/AcalephStorage/consul-alerts/blob/master/notifier/slack-notifier.go#L18

Any idea what's going on ?

I have made a PR to fix the issue: https://github.com/AcalephStorage/consul-alerts/pull/252

[chakravarthy7]

Hello,

In our project we really miss this feature. Here is a proposal on how it can be done.

Named notifiers of a specific type

Currently, it is only possible to have 1 configuration for 1 notifier type, e.g. consul-alerts/config/notifiers/email.

With multiple configurations, it will be possible to send emails to different receivers or post to different slack channels. API example:

• consul-alerts/config/notifiers/emails/<email-notifier-name> - a place to keep named notifiers of type emails
• consul-alerts/config/notifiers/slacks/<slack-notifier-name> - a place to keep named notifiers of type slack

Then, in the notification profiles it should be possible not only to turn on/off the notifier, but also specify a notifier name:

{
  "Interval": 10,
  "NotifList": {
    "log":false,
    "emails": ["team2", "team-3"]
  }
}

Notification Profile Activation using regular expressions

Currently, for every service, check or host to be notified about, there should be a record in consul-alerts/config/notif-selection/[services|nodes|checks]/ directories. However, It is not possible to configure not just one service, but a group of services. To solve this, we can use consul-alerts/config/notif-selection/[services|nodes|checks] keys for the json, mapping groups of services (matched by regexp) to a list of notification profiles. Example:

Key: consul-alerts/config/notif-selection/services Value:

{
  "^team-1-.*$": ["profile-1", "profile-2"]
}

@Gerrrr i am trying to send consul alerts to multiple slack channels through regex pattern that you mentioned above. what i did is i have created new profile under CONSUL-ALERTS/CONFIG/NOTIF-PROFILES/ test as below (already had one profile before this) { "Interval": 1, "NotifList": { "slack": true }, "VarOverrides": { "slack": { "channel": "#channel-name", "enabled": true, "detailed": true, "url": "some url" } } } then change the config under CONSUL-ALERTS/CONFIG/NOTIF-SELECTION/services as below

{ "^service-.*$": ["profile-1", "test"] }

I am still not getting any alerts to my slack channel. Could you tell me, is this right way to do it? do i need to restart consul-alerts or consul in cluster after i make config changes???