search

holos-run / holos

Holos - The Holistic platform manager
https://holos.run
Apache License 2.0
27 stars 0 forks source link

DB Obs - Database focused observability stack #104

Closed jeffmccune closed 7 months ago

jeffmccune commented 7 months ago

In addition to a central obs stack, it might be worthwhile to configure a smaller stack for each stage of each project. Particularly for the IAM login service. To start, focus on the database itself with ZITADEL as the use case.

Crunchy has a nice solution for this:

Refer to the Crunchy Postgres for Kubernetes Monitoring stack.

jeffmccune commented 7 months ago

It was refreshingly straight forward to use the crunchy monitoring example and add the monitoring key to the postgrescluster spec.

Got to the dashboard using k port-forward svc/crunchy-grafana 3000 but it should be straight forward to add a VirtualService and modify the grafana.ini like we do.

Some ideas:

Does it make sense to have an authenticated /holos/grafana path for the environment's host?

How hard is it to configure slack notifications?

Alerts look good, there's a warning for PGDATA at 75% and critical at 90%. Need to test them though.

jeffmccune commented 7 months ago

Closing this for now, we have monitoring merged in. We can revisit how to expose this via the auth proxy in a later ticket.