louislam / uptime-kuma

A fancy self-hosted monitoring tool
https://uptime.kuma.pet
MIT License
60.16k stars 5.38k forks source link

PagerDuty - Option to periodically send resolve events or better handling of resolve events to ensure PagerDuty resolved the incident. #5322

Open invisibleninja06 opened 2 weeks ago

invisibleninja06 commented 2 weeks ago

📑 I have found these related issues/pull requests

I didn't find anything about resolved notifications in relation to PagerDuty.

🏷️ Feature Request Type

Change to existing notification-provider

🔖 Feature description

We sometimes see that some monitors will have become healthy in Uptime-Kuma but were not auto resolved in PagerDuty. Pausing and starting the monitor doesn't seem to send any resolved event, something i think Uptime robot was for us.

Some fallback to resolving these would be ideal as we have people investigate why something is down only to find out its back up in uptime-kuma but not in Pagerduty

✔️ Solution

If possible maybe some way for Uptime-Kuma to validate that the incident it triggered does get closed by its resolved message or an option to Periodicly send resolved events for a PagerDuty notification setup.

❓ Alternatives

Only alternative for us will be to code our own check and manually run that on any open PagerDuty incidents to basically double check if Uptime-Kuma and PagerDuty disagree. I would rather have more faith and resiliency in Uptime-Kuma's ability to resolve the incident with PagerDuty.

📝 Additional Context

We have a lot of monitors on our Uptime-Kuma setup so maybe that's why it sometimes fails to send the resolved notification the first time. Hopping v2 will alleviate that but also thought I would make this feature request as, if indeed Uptime-Kuma only sends a resolve once for PagerDuty, this could be possibly improved.