Stateful Channel Plugins

[julianbrost]

Probably not the best title, so let me try to explain what I mean by that.

It would be nice if notifications for the same incident could be grouped together, so that for example, with e-mail, follow-up notifications show up as a response to the previous e-mail and for a chat, messages for the same incident are within one thread.

In order to achieve this, it's probably necessary to provide some mechanism to notification plugins to keep some state, like for example the Message-ID of the last and/or first e-mail for an incident or some thread reference for other messengers.

The goal of this issue is to evaluate what exactly would be needed to achieve this and suggest how this could be implemented.

[oxzi]

The channels are currently designed to be stateless. They are receiving notification events and contacts, but are unable to handle custom long-lived data without handling their own database. As an example, some channel might link Icinga Notifications with a ticket system and needs to map Icinga Notifications incident IDs to ticket IDs. In the current architecture, such a channel plugin would need to create this database by itself.

Giving channel plugins access to a key-value-store API to store their additionally needed state would ease channel development.

(Originally posted as https://github.com/Icinga/icinga-notifications/issues/232#issue-2403258385)

[oxzi]

At the moment, the RPC communication follows a request-response pattern unidirectional from Icinga Notifications to the channel plugin. Such a change would require altering this architecture, also allowing requests from the channel to Icinga Notifications.

When thinking about the effect of those changes, it would be possible to enable channels to perform further actions. For example, a channel could create an acknowledgement or downtime. This might be useful for a channel interacting with an "interactive" chat protocol, like XMPP, Matrix, or Telegram. Actually, I got the inspiration after watching Maxim Shchukin's Icinga Summit talk "Telegram Bot as a password-free monitoring front-end" about their interactive Telegram bot.

(Originally posted as https://github.com/Icinga/icinga-notifications/issues/232#issuecomment-2238933817)

[oxzi]

When thinking about the effect of those changes, it would be possible to enable channels to perform further actions. For example, a channel could create an acknowledgement or downtime.

By the way, that's an idea that's already floating around. This was also one of the considerations when implementing the RPC mechanism, many existing JSON-RPC implementations were ruled out because they wouldn't allow bidirectional calls.

(Originally posted by @julianbrost as https://github.com/Icinga/icinga-notifications/issues/232#issuecomment-2239411459)