search

google / slo-generator

SLO Generator computes SLIs, SLOs, Error Budgets and Burn Rates from supported backends, then exports an SLO report to supported targets.
Apache License 2.0
482 stars 76 forks source link

[New exporter] Prometheus Remote Write #122

Open dgzlopes opened 3 years ago

dgzlopes commented 3 years ago

The remote write and remote read features of Prometheus allow transparently sending and receiving samples.

Right now slo-generator has support for the Prometheus Push Gateway, but I think having support for Remote Write would be really cool! b/c:

Inspiration: https://pypi.org/project/opentelemetry-exporter-prometheus-remote-write/

ocervell commented 3 years ago

Hi @dgzlopes ! Sounds great, what is the use case for supporting this ? Which backend do you want to exporting to ?

dgzlopes commented 3 years ago

In our case, we're using Grafana Cloud.

To ship the metrics to this "hosted" Prometheus instance we've to use Prometheus Remote Write.

I suppose that I can deploy the PushGateway and collect the metrics the same way I do with the rest of the services, but I thought that maybe skipping this third service was a good idea (fewer things to operate :P).

ocervell commented 3 years ago

Yep, sounds like a good idea. Would you be able to make a PR for it ? Thanks :)

skinlayers commented 1 year ago

@ocervell I've forked @oktocat 's repo that was the source of PR https://github.com/google/slo-generator/pull/177. I've rebased it on the latest master and updated the changes. Unfortunately, the remote write related tests are failing. As I've never worked with mock before, I have been unable to fix the tests. Example:

FAILED tests/unit/test_compute.py::TestCompute::test_export_prometheus_remote_write - RuntimeError: Working outside of application context.

But the code does appear to work when tested against a local Granfana Mimir setup.

I'm going to see if I can get approval to share my the updated code upstream. In the mean time, I'm going to try to implement the changes you requested in the original PR. Let me know if there's anything else I can do to help get this integrated smoothly. :)