search

osism / kolla-operations

Repository for Grafana/Kibana dashboards and Prometheus alerting rules
https://www.osism.tech
9 stars 11 forks source link

Use monitoring-mixins for dashboards and alerts #48

Open janhorstmann opened 1 month ago

janhorstmann commented 1 month ago

Monitoring-mixins ([1]) allow upstream collaboration on alerts and grafana dashboards and convergence on monitoring best practices through bundling mixins with the corresponding software. Downstream customization of alerts and dashboards may be done using jsonnet ([2]). Usage of jsonnet-bundler ([3]) gives a clear reference of where dashboards and alerts originated and which version is used through .src/jsonnetfile.json and .src/jsonnetfile.lock.json.

[1] https://monitoring.mixins.dev/

[2] https://github.com/google/jsonnet

[3] https://github.com/jsonnet-bundler/jsonnet-bundler

janhorstmann commented 1 month ago

If we want to use this, then I would like to also add a check to make sure that checked in alerts and dashboards are in sync. Maybe some of the exisitng dashboards/alerts are redundent now and should be deleted. Still needs testing

janhorstmann commented 1 week ago

Waiting on

https://github.com/prometheus/prometheus/pull/15334 https://github.com/prometheus/node_exporter/pull/3147

in order not to regress on the angular deprecation of grafana