search

prometheus / alertmanager

Prometheus Alertmanager
https://prometheus.io
Apache License 2.0
6.68k stars 2.16k forks source link

alertmanager_notification #3387

Open Zakariaa222 opened 1 year ago

Zakariaa222 commented 1 year ago

Hi, I configured the alertmanager.yml file to send me alerts to my outlook email box, but I don't receive anything. To check I typed in the terminal the following command: smtp.office365.com:587 but it does not pass. What is the solution please

grobinson-grafana commented 1 year ago

Hi! :wave: This isn't enough information to be able to help I'm afraid. Perhaps you can share the logs from your Alertmanager when running with --log.level=debug?

Zakariaa222 commented 1 year ago

Hi, I added the line --log.level=debug in the file "/etc/systemd/system/alertmanager.service" like this :

[Unit] Description=AlertManager for Prometheus Wants=network-online.target After=network-online.target

[Service] User=alertmanager Group=alertmanager Type=simple ExecStart=/usr/local/bin/alertmanager --config.file /etc/alertmanager/alertmanager.yml --log.level=debug

[Install] WantedBy=multi-user.target

After i typed the command journalctl -u alertmanager in the terminal to get the Alertmanager logs and here is the result:

juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.071Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revi> juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.071Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:5> juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.071Z caller=cluster.go:173 level=debug component=cluster msg="resolved peers to following addresses" peers= juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.072Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=19> juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.073Z caller=delegate.go:236 level=debug component=cluster received=NotifyJoin node=01H2DKB1B87YGSEVCY51YFT> juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.073Z caller=cluster.go:266 level=debug component=cluster msg="joined cluster" peers=0 juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.074Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.124Z caller=main.go:395 level=debug externalURL=http://prometheus:9093 juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.124Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/e> juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.125Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration > juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.129Z caller=main.go:441 level=info component=configuration msg="skipping creation of receiver not referenc> juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.129Z caller=main.go:525 level=debug routePrefix=/ juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.131Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 juin 08 14:32:36 prometheus alertmanager[283423]: ts=2023-06-08T13:32:36.131Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 juin 08 14:32:38 prometheus alertmanager[283423]: ts=2023-06-08T13:32:38.074Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 el> juin 08 14:32:40 prometheus alertmanager[283423]: ts=2023-06-08T13:32:40.074Z caller=cluster.go:703 level=debug component=cluster msg="gossip looks settled" elapsed=4.0006136s juin 08 14:32:42 prometheus alertmanager[283423]: ts=2023-06-08T13:32:42.075Z caller=cluster.go:703 level=debug component=cluster msg="gossip looks settled" elapsed=6.0013831s juin 08 14:32:44 prometheus alertmanager[283423]: ts=2023-06-08T13:32:44.075Z caller=cluster.go:703 level=debug component=cluster msg="gossip looks settled" elapsed=8.0017764s

grobinson-grafana commented 1 year ago

Great! You also need to send an alert to the Alertmanger so it attempts to send an email using the configured smtp configuration, otherwise we won't be able to see why it failed.

Zakariaa222 commented 1 year ago

I have an ubuntu machine which I typed in its terminal the following command stress --cpu 4 to increase the cpu usage to 100% so the alertmanager alert will trigger. Here is the alert in the file alert_rules.yml :

And here is the alertmanager.yml file which contains the smtp configuration :

global:

route: group_by: ['alertname'] receiver: 'email_outlook'