Open Shanti-G opened 2 years ago
You made the mistake of using .Labels
, which is part of the Alert
structure.
You may want to use .CommonLabels
at that point?
Check the documentation https://prometheus.io/docs/alerting/latest/notifications/.
Thanks @sbrass for getting back on this.
We have the pdseverity
defined for each alert at Alert
level.
Would you be having any suggestion on getting this alert level data here? Thanks.
The answer depends on your configuration, for example chosen routes and your application case.
I would try to route explicitly by your pdseverity label, i.e., all alerts send to a receiver will share a common label, pdseverity, which you then can access by .CommonLabels.pdseverity
.
Alternatively, if you group alerts by cluster, service and/or host and have multiple alerts with possible differing pdseverity, you will want to choose the highest pdseverity to apply as payload severity? I would expect Go Template to be able to handle such a case, where you want to have the highest severity printed. However, I can not come up with a quick example for such an application.
What did you do? I am trying to set the severity using templated label with below pagerduty configuration in alertmanager:
I am setting
.Labels.pdseverity
on an alert in Prometheus.What did you expect to see? The alertmanager should be able to trigger a PD alert with its severity set based on the templated label.
What did you see instead? Under which circumstances? PagerDuty v2 API allows severity of alert to be set, default being
error
. With the above configuration, though an alert is infiring
state in Prometheus, I don't get the triggered alert in PD.However, when I directly set the value, like below, I get to see the incident in PD with the given severity, i.e. warning:
Would like to know how to set the severity field with a templated value. Thanks in advance.
Environment
Alertmanager version: alertmanager, version 0.21.0 (branch: HEAD, revision: 4c6c03ebfe21009c546e4d1e9b92c371d67c021d) build user: root@dee35927357f build date: 20200617-08:54:02 go version: go1.14.4
Prometheus version: prometheus, version 2.27.1 (branch: HEAD, revision: db7f0bcec27bd8aeebad6b08ac849516efa9ae02) build user: root@fd804fbd4f25 build date: 20210518-14:17:54 go version: go1.16.4 platform: linux/amd64
Logs: No error logs from alertmanager to indicate configuration error.