matofeder
commented
6 months ago There are at least 3 issues we have to tackle if we want to adapt the current Observability solution to be able to monitor k3s clusters:
- K3s control plane components kube-scheduler, kube-proxy, and kube-controller do not expose metrics endpoints (by default)
Those metrics endpoints should be enabled, see https://github.com/k3s-io/k3s/issues/3619#issuecomment-878501106
- This configuration should be part of k3s deployment, hence there is nothing to do on the Observability side, just update the docs to inform k3s users to do so
- kube-prometheus-stack helm values should be adjusted to monitor endpoints outside the k8s overlay network, because the k3s runs as a single process on the host, see see https://github.com/k3s-io/k3s/issues/3619#issuecomment-878501106
- K3s is exposing metrics from all CP components in each of them. So we will get duplicated data in the Prometheus. see https://github.com/k3s-io/k3s/issues/3619, try workaround mentioned here https://github.com/prometheus-community/helm-charts/issues/2865#issuecomment-1490318965
- This configuration should be part of k3s deployment, hence there is nothing to do on the Observability side, just update the docs to inform k3s users to do so
As a CSP I want to utilize my IaaS management k3s cluster with SCS Observability solution, and observe the k3s cluster itself as well as the IaaS infrastructure around.