search

tomkerkhove / promitor

Bringing Azure Monitor metrics where you need them.
https://promitor.io
MIT License
249 stars 91 forks source link

Provide metric sink for Atlassian Opsgenie #1013

Closed tomkerkhove closed 3 years ago

tomkerkhove commented 4 years ago

Provide metric sink for Atlassian Opsgenie to report metrics.

adamconnelly commented 4 years ago

Hey Tom - just noticed this one and I wanted to double check something. Alertmanager already has support for sending alerts to OpsGenie, and has support for generic webhooks. So the way that works is that Prometheus scrapes Promitor, stores the metrics and evaluates any alerting rules, Prometheus notifies Alertmanager, which then notifies OpsGenie / Slack / whatever.

So it's kinda like "Azure Monitor <-- Promitor <-- Prometheus --> Alertmanager --> OpsGenie".

I'm a little out the loop, so I'm not completely sure where you're going with the sinks, so maybe you're doing it for a different purpose, but I just wanted to point that out in case you haven't used Alertmanager.

tomkerkhove commented 4 years ago

The idea is to see twofold actually:

  1. Atlassian Statuspage allows you to write custom metrics
  2. See if it would make sense to emit info to Atlassian Opsgenie to avoid the need for Prometheus & Alertmanager in general.

This issue is mainly for the first scenario, but need to dig in Opsgenie as I haven't used it before.

You can find a small sample about Statuspage here. Are you using this?

adamconnelly commented 4 years ago

Ok - I get you. At ResDiary we use Statuspage (status.resdiary.com) and OpsGenie. We don't currently push any custom metrics to Statuspage so I can't really comment on that just now, although it sounds interesting.

The only thing I'd say about your second scenario is it looks like OpsGenie supports integrating with Azure Monitor directly (https://docs.opsgenie.com/docs/microsoft-azure-integration), so in the situation where you're using Azure metrics without Prometheus, would you not just go straight from Azure Monitor into OpsGenie?

Our use-case for using Promitor is so that it all goes into our Prometheus metrics stack so we can write Prometheus alert rules as code and deploy them as part of our applications. It means that we can use the same syntax to create alerts that we use for alerting based on other Prometheus metrics.

tomkerkhove commented 4 years ago

I'll focus on Atlassian Statuspage for now to integrate metrics there in case Opsgenie is not being used

tomkerkhove commented 4 years ago

Adding Atlassian Statuspage as part of #1152