KlavsKlavsen
commented
4 years ago Found the cause.. as my URL had an ending / - it ended up with a // - which you can see returns a 301.. and then it tries on the new URL- but without the POST data - which is why the silence does not get created. Plugin should clearly fail when it does not have an id - and it should also try and remove any ending / - in alertmanager api url given - to avoid this issue.
Issue Summary When I ack an alert in Alerta - I get this output (with debug enabled):
2020-07-30 09:51:19,620 alerta.plugins.prometheus[33]: [DEBUG] Alertmanager: Add silence for alertname=InstanceDown instance=192.168.26.146:63512 timeout=86400 request_id=d3534a96-ee4c-4065-a26e-eacf323d745e ip=192.168.20.1
2020-07-30 09:51:19,620 alerta.plugins.prometheus[33]: [DEBUG] Alertmanager: URL=http://api-prometheus-klavs2-alertmanager//api/v1/silences request_id=d3534a96-ee4c-4065-a26e-eacf323d745e ip=192.168.20.1
2020-07-30 09:51:19,632 urllib3.connectionpool[33]: [DEBUG] http://api-prometheus-klavs2-alertmanager:80 "POST //api/v1/silences HTTP/1.1" 301 0 request_id=d3534a96-ee4c-4065-a26e-eacf323d745e ip=192.168.20.1
2020-07-30 09:51:19,637 urllib3.connectionpool[33]: [DEBUG] http://api-prometheus-klavs2-alertmanager:80 "GET /api/v1/silences HTTP/1.1" 200 30 request_id=d3534a96-ee4c-4065-a ││ 26e-eacf323d745e ip=192.168.20.1
2020-07-30 09:51:19,638 alerta.plugins.prometheus[33]: [DEBUG] Alertmanager: Added silenceId unknown to attributes request_id=d3534a96-ee4c-4065-a26e-eacf323d745e ip=192.168.2 ││ 0.1
update_time='2020-07-30T09:51:19.616Z', history=(ARRAY[('30ceed40-1ef1-4179-9745-78df6ffc1ea2', 'InstanceDown', 'major', 'ack', NULL, 'ack operator action ( silenced in Alertmanager)', 'ack', '2020-07-30T09:51:19.616Z'::timestamp, 'zxxx@xxc.dk')::history] || history)[1:100]
But there are no silences in Alertmanager created :(
Environment