search

JacksonWrath / k8s-config-weebcluster

This repo aims to define the state of my primary cluster.
MIT License
0 stars 0 forks source link

Set up Loki again #7

Closed JacksonWrath closed 9 months ago

JacksonWrath commented 1 year ago

Haven't done this since I migrated from my old k8s cluster to the new one.

Look into using a separate grafana instance, since I want to add other things to that

JacksonWrath commented 11 months ago

Loki now has a simpler scalable mode, in between monolithic and full microservices: https://grafana.com/docs/loki/latest/fundamentals/architecture/deployment-modes/

Their Helm chart deploys in this mode, so I just did that.

Grafana deployed separately via Helm chart.

Promtail deployed via Helm; by default it uses k8s service discovery. Added config to scrape host systemd journal as well.

JacksonWrath commented 9 months ago

Re-opened. It appears that it's deleting logs older than a couple hours, even though retention is set to "forever".

"Simple scalable" mode seems to have issues in general looking through the Loki repo, so I need to just do the full setup.

Practice with Tanka/jsonnet will be good anyway, since I kind wanna move to that instead of managing a bunch of yaml.

JacksonWrath commented 9 months ago

Done.