Create/adapt dashboards covering major Coder architectural components

[dannykopping]

• [x] Controlplane
  • [x] Number of replicas
  • [x] API requests per route per replica
  • [x] CPU per replica (requests, limits, usage)
  • [x] Memory per replica (requests, limits, heap, RSS, gc pacer)
  • [x] Pod restarts
  • [x] Pubsub latency
  • [x] Active users vs license seats
• [ ] Workspaces
  • [x] Pod restarts
  • [x] Pod OOMs
  • [ ] Pods with fullest disks (requires https://github.com/kubernetes-sigs/metrics-server/ to be installed)
  • [x] Pods with highest CPU usage
  • [x] Pods with highest memory usage
• [x] Workspaces Detail (inspiration)
  • [x] Logs for workspace
  • [x] Template name & version
  • [x] Owner
  • [x] Build status
• [ ] Workspace proxies
  • [ ] Status
• [x] Provisioner
  • [x] Number of built-in provisioners
  • [x] Number of external provisioners
  • [x] Build successes vs failures
  • [x] Build successes vs failures (per template)
  • [x] Builds in progress
• [ ] Networking
  • [ ] Websocket
  • [ ] Current connections
  • [ ] Traffic by workspaces
  • [ ] Traffic by coderd replica
  • [ ] Agent
  • [ ] Current connections
  • [ ] Traffic by workspaces
  • [ ] Traffic by coderd replica
• [ ] Health checks (stretch)

[dannykopping]

Make sure we include workspace OOMs in one of the dashboards: https://github.com/coder/customers/issues/584#issuecomment-2093566479

[bpmct]

@dannykopping Can we announce the beta without these remaining dashboards and move these over to separate issues in coder/observability? Asking because we have a lot to cover over in June (notifications, etc) and customer feedback can influence our priorities for the observability helm chart.

[dannykopping]

I would argue so, yeah 👍