Improve new control plane dashboards

[metalmatze]

With #205 merged we have a few new dashboards for the control plane (apiserver, scheduler, proxy, kubelet).

Here are a few TODOs outline for the future:

• [ ] We should unclutter the names and separate components and workload dashboards more.
• [ ] We should make sure that components alerts are represented in dashboards. Example: KubeAPIErrorsHigh needs to be visible in the apiserver dashboard. Reuse recording rule.
• [ ] Reuse more recording rules for control plane dashboards (lots of similar queries across dashboards).
• [ ] Go metrics about components should probably be separated. Either own dashboard or no need at all? Let's discuss.
• [ ] Add more from the discussion below

/cc @povilasv @brancz

[brancz]

Incoming/Outgoing HTTP requests differentiation is not super obvious right now, I think explicitly labeling those would be good.

[povilasv]

Go metrics about components should probably be separated. Either own dashboard or no need at all? Let's discuss.

I find those really useful when you are debugging a control plane failure, OOMs / crashloops etc.

The way I envisioned this is that SREs get a single view where they look for symptoms, like "cpu is going thru the roof and we are getting tons of requests, " type of deal.

Agreed on all other points.

[github-actions[bot]]

This issue has not had any activity in the past 30 days, so the stale label has been added to it.

• The stale label will be removed if there is new activity
• The issue will be closed in 7 days if there is no new activity
• Add the keepalive label to exempt this issue from the stale check action

Thank you for your contributions!