search

SovereignCloudStack / k8s-observability

Deployment manifests and knowledge base for the KaaS observability solution
https://scs.community/
Apache License 2.0
1 stars 0 forks source link

Enable Loki on monitoring.scs.community #50

Closed jschoone closed 4 months ago

jschoone commented 6 months ago

As an SCS Operator I want to expose my Kubernetes logs to a Prometheus, so I can make this logs accessible without giving permissions in my Kubernetes clusters.

In SCS we now have a specific case with the shared Kubernetes clusters. For example we want to give users permissions to test Cluster Stacks on it without giving them access to the CSO/CSPO-Resources directly on the cluster. But since they need/want to now what's wrong if the cluster doesn't come up, they can check the logs on monitoring.scs.community.

matofeder commented 4 months ago

dNation monitoring needed some Loki related updates.

See https://github.com/dNationCloud/kubernetes-monitoring-stack/pull/100

matofeder commented 4 months ago

Loki was deployed at https://monitoring.scs.community/ along with the Promtail agent on each Observer node.

This setup ensures that the SCS Observer platform collects logs from itself and can also serve as a global, horizontally scalable, highly available, multi-tenant log aggregation system.

Now we are able so see logs and metrics from observer cluster side-by side, e.g. see some container logs and metrics here or via the grafana explore https://monitoring.scs.community/explore

Find en example helm values in values-observer-scs.yaml.

Contact @dnation folks for productive values used for SCS instance deployment available at monitoring.scs.community.

Loki API endpoint for logs ingestion is available at monitoring.scs.community/loki. Basicauth is used here, contact @dnation folks for user/pass info.

matofeder commented 4 months ago

Several follow-up issues have been opened: