gmauleon / alertmanager-zabbix-webhook

Prometheus alertmanager webhook that sends alerts to a Zabbix server via trapper items.
GNU General Public License v3.0
62 stars 30 forks source link

webhook response status 400 bad request #1

Closed ricklagerweij closed 4 years ago

ricklagerweij commented 6 years ago

Hi @gmauleon,

Currently configuring the alert from prometheus to the webhook but I'm getting a 400 bad request.

Alertmanager logs; level=debug ts=2018-09-20T11:53:57.096317774Z caller=notify.go:668 component=dispatcher msg="Notify attempt failed" attempt=1 integration=webhook receiver=zabbix err="unexpected status code 400 from http://10.72.52.106:8080/alerts"

Is the zabbix-alertmanager-webnhook still working properly with the latest alertmanager?

ricklagerweij commented 6 years ago

@gmauleon missing some of the json formats e..g.

{ "version": "4", "groupKey": , // key identifying the group of alerts (e.g. to deduplicate) "status": "<resolved|firing>", "receiver": , "groupLabels": , "commonLabels": , "commonAnnotations": , "externalURL": , // backlink to the Alertmanager. "alerts": [ { "status": "<resolved|firing>", "labels": , "annotations": , "startsAt": "", "endsAt": "", "generatorURL": // identifies the entity that caused the alert }, ... ] }

gmauleon commented 6 years ago

Thanks for the heads-up, I'll try to have a look later this week

ricklagerweij commented 6 years ago

@gmauleon we fixed it, works in my fork thanks anyway;)

gmauleon commented 6 years ago

Hum actually I think you probably got lucky there, I don't think the missing fields are the problem. It's the way I was handling alerts as a group only instead of individual alerts which is completely wrong :)

I bet if you fire up multiple alerts at the same time you'll have the problem again.

gmauleon commented 6 years ago

@ricklagerweij if you ever have time, try out the last release, should handle multi alerts the way it always should have.

ricklagerweij commented 6 years ago

Awesome @gmauleon! We did indeed get issues by sending multiple alerts, solved by your latest commit :)

Currently the alertmanager only sends data to the webhook when a trigger hits it threshold, how would it be possible in any way to send a 0 value to zabbix when no data is send from alertmanager?

gmauleon commented 6 years ago

Hum, I don't think it's really the job of an alertmanager webhook though, it's purpose is only to forward alerts to whatever backend it supports. You could handle that case direcly at the source, like having different rules for when Prometheus can't scrap a target or when there is absent data there and have a different item in Zabbix for those.

pauloeliasjr commented 5 years ago

Hi @gmauleon

I'm still getting 400 when trying to test the webhook. I'm using it on kubernetes (latest image tag). Could you please give me some directions to get it working?

gmauleon commented 5 years ago

Hi there, Sorry I'm not looking much at this project anymore those days. I`ll try to have a look with the latest alertmanager somewhere this week, see if there is a quick win.

gmauleon commented 5 years ago

Hey @pauloeliasjr,

Just gave it a try with the latest alertmanager and I don't see any problems. Do you have more context for your 400? Any logs from the webhook? The configuration you are using might help too.

Thanks Gael