timescale / tobs

tobs - The Observability Stack for Kubernetes. Easy install of a full observability stack into a k8s cluster with Helm charts.
Apache License 2.0
560 stars 61 forks source link

Restructure collecting of prometheus metrics and bump dependencies #606

Closed paulfantom closed 1 year ago

paulfantom commented 1 year ago

What this PR does / why we need it

This PR aims to reduce learning curve and simplify UX by configuring prometheus instances shipped with tobs to gather all PrometheusRules, ServiceMonitors, PodMonitors, and Probes that are injected into the cluster. This way the observability platform can collect data from the whole cluster.

To opt-out a resource from being used by prometheus shipped with tobs, you need to label it with tobs/excluded label (value doesn't matter).

This PR is also disabling scraping metrics using prometheus label mechanism and forcing users to use prometheus-operator CRs. This should prevent data duplication in larger clusters and align us closer to kube-prometheus project (jsonnet one). Feature can be re-enable if needed by the user.

Checklist

paulfantom commented 1 year ago

I think I found the main issues preventing merge of this PR: