search

alerta / alerta-contrib

Contributed integrations, plugins and custom webhooks
http://alerta.io
MIT License
119 stars 170 forks source link

Annotations from Prometheus/Grafana are not passed over/ raw data is null #394

Open RedLightLink opened 1 year ago

RedLightLink commented 1 year ago

Issue Summary Hello, I have a problem with the forwarder plug in, the rawData is sent as null as seen below.

Package sent from prometheus webhook:

POST /api/webhook/prometheus HTTP/1.1 Host: home User-Agent: Grafana Content-Length: 1969 Content-Type: application/json Accept-Encoding: gzip

{"receiver":"kafka","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"Panel Title","grafana_folder":"test"},"annotations":{},"startsAt":"2023-05-10T07:59:00Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://prometheus:3000","fingerprint":"dc2402f707754f2d","silenceURL":"http://prometheus:3000","dashboardURL":"http://prometheus:3000","panelURL":"http://prometheus:3000","values":{"B0":16648},"valueString":"[ var='B0' metric='Value' labels={name=test_random_graph2, group=test, instance=172.xx.xx.xxx:8000, job=mon_TEST, platform=TEST, service=testpr} value=16648 ]"}],"groupLabels":{"alertname":"Panel Title","grafana_folder":"test"},"commonLabels":{"alertname":"Panel Title","grafana_folder":"test"},"commonAnnotations":{},"externalURL":"http://prometheus:3000","version":"1","groupKey":"{}:{alertname=\"Panel Title\", grafana_folder=\"test\"}","truncatedAlerts":0,"orgId":1,"title":"[FIRING:1] Panel Title test ","state":"alerting","message":"Firing\n\nValue: B0=16648\nLabels:\n - alertname = Panel Title\n - grafana_folder = test\nAnnotations:\nSource: http://prometheus:3000: http://prometheus:3000: http://prometetheus:3000 http://prometheus:3000"}HTTP/1.1 201 CREATED Server: nginx Date: Wed, 10 May 2023 07:59:35 GMT Content-Type: application/json Content-Length: 993 Connection: keep-alive Access-Control-Allow-Origin: http://localhost Vary: Origin, Accept-Encoding Content-Encoding: gzip X-Request-ID: a042be2e-8d08-40c9-ae1a-68f354db7a40

{"alert": {"attributes": {"endsAt": "0001-01-01T00:00:00Z", "externalUrl": "http://prometheus:3000/", "moreInfo": "<a href=\"http://prometheus:3000Id=1\" target=\"_blank\">Prometheus Graph", "startsAt": "2023-05-10T07:59:00Z"}, "correlate": [], "createTime": "2023-05-10T07:59:35.032Z", "customer": "Home", "duplicateCount": 0, "environment": "Production", "event": "Panel Title", "group": "Prometheus", "history": [{"event": "Panel Title", "href": "http://webalarm", "id": "337323ff-b493-4dc1-92ec-ef7ffc62e1b4", "severity": "warning", "status": "open", "text": "WARNING: n/a is Panel Title", "timeout": 86400, "type": "new", "updateTime": "2023-05-10T07:59:35.032Z", "user": "user", "value": null}], "href": "http://webalert", "id": "337323ff-b493-4dc1-92ec-ef7ffc62e1b4", "lastReceiveId": "337323ff-b493-4dc1-92ec-ef7ffc62e1b4", "lastReceiveTime": "2023-05-10T07:59:35.037Z", "origin": "prometheus/-", "previousSeverity": "indeterminate", "rawData": "{\"status\": \"firing\", \"labels\": {\"alertname\": \"Panel Title\", \"grafana_folder\": \"test\"}, \"annotations\": {}, \"startsAt\": \"2023-05-10T07:59:00Z\", \"endsAt\": \"0001-01-01T00:00:00Z\", \"generatorURL\": \"http://prometheus:3000", \"fingerprint\": \"dc2402f707754f2d\", \"silenceURL\": \"http://prometheus:3000", \"dashboardURL\": \"http://prometheus:3000\", \"panelURL\": \"http://prometheus:3000", \"values\": {\"B0\": 16648}, \"valueString\": \"[ var='B0' metric='Value' labels={name=test_random_graph2, group=test, instance=172.xx.xx.xxx:8000, job=mon_TEST, platform=TEST, service=testpr} value=16648 ]\"}", "receiveTime": "2023-05-10T07:59:35.037Z", "repeat": false, "resource": "n/a", "service": [""], "severity": "warning", "status": "open", "tags": ["grafana_folder=test"], "text": "WARNING: n/a is Panel Title", "timeout": 86400, "trendIndication": "moreSevere", "type": "prometheusAlert", "updateTime": "2023-05-10T07:59:35.037Z", "value": null}, "id": "337323ff-b493-4dc1-92ec-ef7ffc62e1b4", "status": "ok"}

Package sent from plug in:

POST /alert HTTP/1.1 Host: webalarm sentry-trace: bf25eed8db9a4c4eb0523885e5ce2be0-853e085ae7d2d37f-0 baggage: sentry-trace_id=bf25eed8db9a4c4eb0523885e5ce2be0,sentry-environment=production,sentry-release=9.0.0,sentry-transaction=webhooks.custom User-Agent: python-requests/2.28.1 Accept-Encoding: gzip, deflate, br Accept: / Connection: keep-alive X-Alerta-Loop: http://webalarm:80 X-Request-ID: c46f07bb-582e-4e52-8dd4-8e97565e246e Content-Type: application/json Content-Length: 757 Authorization: Basic Zm9vOmJhcg==

{"id": "337323ff-b493-4dc1-92ec-ef7ffc62e1b4", "resource": "n/a", "event": "Panel Title", "environment": "Production", "severity": "warning", "correlate": [], "service": [""], "group": "Prometheus", "value": null, "text": "WARNING: n/a is Panel Title", "tags": ["grafana_folder=test"], "attributes": {"endsAt": "0001-01-01T00:00:00Z", "moreInfo": "<a href=\"http://prometheus:3000" target=\"_blank\">Prometheus Graph", "startsAt": "2023-05-10T07:59:00Z", "externalUrl": "http://prometheus:3000/"}, "origin": "prometheus/-", "type": "prometheusAlert", "createTime": "2023-05-10T07:59:35.052Z", "timeout": 86400, "rawData": null, "customer": "AlertManager"}HTTP/1.1 200 OK Date: Wed, 10 May 2023 07:59:35 GMT Server: Apache/2.4.54 (Debian) X-Powered-By: PHP/7.4.30 Vary: Accept-Encoding Content-Encoding: gzip Content-Length: 237 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/html; charset=UTF-8

Environment

Additional context I updated from an older version of alerta and this started to happen after the update.