Remove Loki (or fix the compactor crashloop)

prometheus / test-infra

Prometheus E2E benchmarking tool

Apache License 2.0

157 stars 67 forks source link

Remove Loki (or fix the compactor crashloop) #780

Open bwplotka opened 4 days ago

bwplotka commented 4 days ago

Compactor is crashlooping for Loki

Seems like a configuration issue, but I wonder if we are using Loki for Prombench use cases. We run in GKE and logging is already OOTB in, so it feels we could save some cost by removing Loki for our use case:

Disclaimer: I am working for Google Cloud, proposing to remove Grafana Loki instance from prombench, so second opinion from non-Googler Prometheus team member would be nice here (((((:

cc @bboreham @kakkoyun are you using Loki?

bboreham commented 4 days ago

I fixed up the config. https://github.com/prometheus/test-infra/pull/781

I find Google Cloud logging terrible to use compared to Loki. Maybe it's a case of what you are used to.

E.g. it decorates everything with an angry red mark

bboreham commented 4 days ago

More generally, the test-infra repo is set up to be usable in different situations - GCP, AWS, Kind. It would simplify quite a few things if we hard-coded to work only at Google.