search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.62k stars 8.22k forks source link

kibana push alerts to alertmanager #124921

Open zfrhv opened 2 years ago

zfrhv commented 2 years ago

Describe the feature: add option for the kibana to push alerts to prometheus AlertManager.

assuming we have kubernetes/openshift or any other applications. we have the monitoring with prometheus that collects metrics creates alerts and pushes to AlertManager, and we have the elk or efk that collects the logs, and has the option to create alerts. it would be nice to have 1 central point where i could watch and manage all of my alerts. for example alertmanager or any other application that can store alerts.

Describe a specific use case for the feature: being able to see and manage (send email notification, aknowlage alerts...) all of my alerts from AlertManager or any other alerts managing application (sorry i dont have example for other applications because i dont know any). instead of configuring alertmanager seperatly and kibana seperatly. it will be also good if i will have few kibanas, for example if i have multiple clusters, i could store all of my efk alerts in 1 point.

elasticmachine commented 2 years ago

Pinging @elastic/response-ops (Team:ResponseOps)

julienlau commented 1 year ago

FYI this connector does the opposite : https://github.com/webdevops/alertmanager2es

andrianjardan commented 1 week ago

Bump, would be great if this is added.

heespi commented 1 week ago

Thanks for the ping, @andrianjardan ...

Is this integration something that could be addressed via REST using the WebHook connector ?

CC: @mikecote @cnasikas

hartfordfive commented 1 week ago

This could likely be accomplished via the webhook as alertmanager has an HTTP endpoint to trigger alerts. I also completely understand the logic of centralizing all active alerts although it's advised to only have Prometheus handle triggering the alerts to alertmanager (see docs).

With that in mind, if you still choose to go forward with this option, just keep in mind they need to be sent continuously (at most every minute or so I believe) to every alertmanager host in the cluster. When an alert is active in Prometheus, it's sent to all alertmanager hosts on every evaluation interval. Otherwise once AM stops receiving the notifications, it will eventually treat the alerts as resolved. You can find the relevant specs for the alertmanager API here. You can also find a useful article of alertmanager below: