Metrics: Kube API Server

[jchauncey]

https://github.com/kubernetes/kubernetes/blob/f91101e0926acf70864dc83de4a1c680b4bf26cb/pkg/metrics/api_server_metrics.go

We should collect the following metrics from the kube api server:

• apiserver_request_count
• apiserver_request_latencies_bucket
• apiserver_request_latencies_count
• apiserver_request_latencies_sum
• apiserver_request_latencies_summary
• apiserver_request_latencies_summary_count
• apiserver_request_latencies_summary_sum
• etcd_helper_cache_entry_count
• etcd_helper_cache_hit_count
• etcd_helper_cache_miss_count
• etcd_request_cache_add_latencies_summary
• etcd_request_cache_add_latencies_summary_count
• etcd_request_cache_add_latencies_summary_sum
• etcd_request_cache_get_latencies_summary
• etcd_request_cache_get_latencies_summary_count
• etcd_request_cache_get_latencies_summary_sum
• etcd_request_latencies_summary
• etcd_request_latencies_summary_count
• etcd_request_latencies_summary_sum
• rest_client_request_latency_microseconds
• rest_client_request_latency_microseconds_count
• rest_client_request_latency_microseconds_sum
• rest_client_request_status_codes
  They use prometheus to expose this metrics so we should be able to use the telegraf prometheus client to poll that endpoint.

[jchauncey]

You can access the prometheus endpoint for the api server metrics by doing the following:

$ curl https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERIVCE_PORT/metrics --header "Authorization: Bearer $TOKEN" --insecure

Where $TOKEN is the service account token located at /var/run/secrets/kubernetes.io/serviceaccount/token