search

kubernetes-sigs / metrics-server

Scalable and efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.
https://kubernetes.io/docs/tasks/debug-application-cluster/resource-metrics-pipeline/
Apache License 2.0
5.77k stars 1.86k forks source link

Unable to get CPU for container discarding data: missing cpu usage metric #207

Closed khteh closed 4 years ago

khteh commented 5 years ago

I added the following to the 1.8+ manifest and deploy it to EKS.

imagePullPolicy: Always
        command:
        - /metrics-server
        - --kubelet-insecure-tls
        - --kubelet-preferred-address-types=InternalIP

Yet, I see the following errors:

I0123 02:04:30.824820       1 serving.go:273] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
W0123 02:04:31.420786       1 authentication.go:166] cluster doesn't provide client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication to extension api-server won't work.
W0123 02:04:31.448142       1 authentication.go:210] cluster doesn't provide client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication to extension api-server won't work.
[restful] 2019/01/23 02:04:31 log.go:33: [restful/swagger] listing is available at https://:443/swaggerapi
[restful] 2019/01/23 02:04:31 log.go:33: [restful/swagger] https://:443/swaggerui/ is mapped to folder /swagger-ui/
I0123 02:04:31.493567       1 serve.go:96] Serving securely on [::]:443
E0123 02:04:34.290945       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-converse-1: no metrics known for pod
E0123 02:04:34.290964       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-converse-0: no metrics known for pod
E0123 02:05:04.301881       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-converse-1: no metrics known for pod
E0123 02:05:04.301998       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-converse-0: no metrics known for pod
E0123 03:11:31.654059       1 manager.go:102] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:ip-10-0-1-40.ap-southeast-1.compute.internal: unable to get CPU for container "iconverse-nlp" in pod default/iconverse-nlp-1 on node "10.0.1.40", discarding data: missing cpu usage metric, unable to fully scrape metrics from source kubelet_summary:ip-10-0-0-168.ap-southeast-1.compute.internal: unable to get CPU for container "iconverse-converse" in pod default/iconverse-converse-0 on node "10.0.0.168", discarding data: missing cpu usage metric]
E0123 03:11:35.994020       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-converse-0: no metrics known for pod
E0123 03:12:06.018005       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-converse-0: no metrics known for pod
E0123 04:31:31.563306       1 manager.go:102] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:ip-10-0-2-172.ap-southeast-1.compute.internal: unable to get CPU for container "iconverse-nlp" in pod default/iconverse-nlp-0 on node "10.0.2.172", discarding data: missing cpu usage metric
$ k --kubeconfig=$KUBECONFIG get hpa
NAME                 REFERENCE                        TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
iconverse-converse   StatefulSet/iconverse-converse   <unknown>/50%   2         5         2          4h
$ k --kubeconfig=$KUBECONFIG describe hpa iconverse-converse
Error from server (NotFound): the server could not find the requested resource

This is my hpa manifest:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: myapplication
  namespace: default 
spec:
  maxReplicas: 5
  minReplicas: 2
  scaleTargetRef:
    apiVersion: apps/v1 
    kind: StatefulSet
    name: myapplication
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 50
  - type: Resource
    resource:
      name: memory
      targetAverageUtilization: 50

Any insight and advice is appreciated.

wang510918 commented 5 years ago

maybe you didnt provide ca.crt and ca.key to the node of your cluster

khteh commented 5 years ago

Can you provide more details please? Thanks.

mabushey commented 5 years ago

There are 200 forks, does anyone know which (if any) of them fix metric server?

zouyee commented 5 years ago

https://github.com/kubernetes-incubator/metrics-server/issues/212#issuecomment-459321884

khteh commented 5 years ago

Thanks @zouyee. I changed from using "args" to "commands" and I don't see the 401 Unauthorized now. However, kubectl logs -f metrics-server... -n kube-system still shows "no metrics known for pod":

$ k logs -f metrics-server-68df9fbc9f-dgr8v -n kube-system
I0312 03:55:41.841800       1 serving.go:273] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
W0312 03:55:42.433339       1 authentication.go:166] cluster doesn't provide client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication to extension api-server won't work.
W0312 03:55:42.439873       1 authentication.go:210] cluster doesn't provide client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication to extension api-server won't work.
[restful] 2019/03/12 03:55:42 log.go:33: [restful/swagger] listing is available at https://:443/swaggerapi
[restful] 2019/03/12 03:55:42 log.go:33: [restful/swagger] https://:443/swaggerui/ is mapped to folder /swagger-ui/
I0312 03:55:42.488139       1 serve.go:96] Serving securely on [::]:443
E0312 03:55:46.554516       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-nlp-0: no metrics known for pod
E0312 03:55:46.554540       1 reststorage.go:144] unable to fetch pod metrics for pod default/iconverse-nlp-1: no metrics known for pod
  E0312 05:08:42.634201       1 manager.go:102] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:ip-192-168-84-18.ap-southeast-1.compute.internal: [unable to get CPU for container "iconverse-connector" in pod default/iconverse-connector-0 on node "192.168.84.18", discarding data: missing cpu usage metric, unable to get CPU for container "iconverse-fluentd" in pod default/iconverse-connector-0 on node "192.168.84.18", discarding data: missing cpu usage metric], unable to fully scrape metrics from source kubelet_summary:ip-192-168-22-244.ap-southeast-1.compute.internal: [unable to get CPU for container "iconverse-fluentd" in pod default/iconverse-converse-0 on node "192.168.22.244", discarding data: missing cpu usage metric, unable to get CPU for container "iconverse-converse" in pod default/iconverse-converse-0 on node "192.168.22.244", discarding data: missing cpu usage metric, unable to get CPU for container "iconverse-fluentd" in pod default/iconverse-admin-0 on node "192.168.22.244", discarding data: missing cpu usage metric, unable to get CPU for container "iconverse-admin" in pod default/iconverse-admin-0 on node "192.168.22.244", discarding data: missing cpu usage metric, unable to get CPU for container "iconverse-ui" in pod default/iconverse-ui-0 on node "192.168.22.244", discarding data: missing cpu usage metric]]
NKUCodingCat commented 5 years ago

Same here in K8s 1.13.5

cjingtao-a01:metrics-server-0.3.2 cjingtao$ KUBECONFIG=/Users/cjingtao/Documents/GitHub/deploy/Server-End\ Complex/dev/Dev-CD.kubeconfig.yml kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.5", GitCommit:"2166946f41b36dea2c4626f90a77706f426cdea2", GitTreeState:"clean", BuildDate:"2019-03-25T15:19:22Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
cjingtao-a01:metrics-server-0.3.2 cjingtao$ KUBECONFIG=/Users/cjingtao/Documents/GitHub/deploy/Server-End\ Complex/dev/Dev-CD.kubeconfig.yml kubectl logs metrics-server-68d85f76bb-8bb9p -n kube-system
I0505 02:06:44.758400       1 serving.go:273] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
[restful] 2019/05/05 02:06:45 log.go:33: [restful/swagger] listing is available at https://:443/swaggerapi
[restful] 2019/05/05 02:06:45 log.go:33: [restful/swagger] https://:443/swaggerui/ is mapped to folder /swagger-ui/
I0505 02:06:45.133211       1 serve.go:96] Serving securely on [::]:443
E0505 02:07:14.671030       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.je5voly0": no metrics known for node
E0505 02:07:14.671049       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.t2mzj8lv": no metrics known for node
E0505 02:07:14.671053       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.4q1e2xzl": no metrics known for node
E0505 02:07:14.671056       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.4iqiwkgz": no metrics known for node
E0505 02:07:14.671060       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.vrdnkjkv": no metrics known for node
E0505 02:07:14.671063       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.kkvwkpqv": no metrics known for node
E0505 02:07:14.671066       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.3wjheffw": no metrics known for node
E0505 02:07:14.671069       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.hxqlgl0o": no metrics known for node
E0505 02:07:14.671072       1 reststorage.go:129] unable to fetch node metrics for node "kube-worker.r85r2n3n": no metrics known for node
E0505 02:07:19.380532       1 reststorage.go:144] unable to fetch pod metrics for pod default/kser-launcher-1-6b4bd7d8cc-gl8v6: no metrics known for pod
E0505 02:07:19.380547       1 reststorage.go:144] unable to fetch pod metrics for pod default/kser-launcher-1-6b4bd7d8cc-s6bzs: no metrics known for pod
E0505 02:07:19.380551       1 reststorage.go:144] unable to fetch pod metrics for pod default/kser-launcher-1-6b4bd7d8cc-nxpvn: no metrics known for pod
E0505 02:07:19.380554       1 reststorage.go:144] unable to fetch pod metrics for pod default/kser-se-wrapper-1-64c978ccbc-q5btm: no metrics known for pod
cjingtao-a01:metrics-server-0.3.2 cjingtao$ KUBECONFIG=/Users/cjingtao/Documents/GitHub/deploy/Server-End\ Complex/dev/Dev-CD.kubeconfig.yml kubectl get pods metrics-server-68d85f76bb-8bb9p -n kube-system -o yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2019-05-05T02:06:41Z"
  generateName: metrics-server-68d85f76bb-
  labels:
    k8s-app: metrics-server
    pod-template-hash: 68d85f76bb
  name: metrics-server-68d85f76bb-8bb9p
  namespace: kube-system
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: metrics-server-68d85f76bb
    uid: 6c661bc4-6eda-11e9-a6de-020061cb48ef
  resourceVersion: "5736885"
  selfLink: /api/v1/namespaces/kube-system/pods/metrics-server-68d85f76bb-8bb9p
  uid: 6c6927a2-6eda-11e9-a6de-020061cb48ef
spec:
  containers:
  - command:
    - /metrics-server
    - --kubelet-insecure-tls
    - --kubelet-preferred-address-types=InternalIP
    image: k8s.gcr.io/metrics-server-amd64:v0.3.1
    imagePullPolicy: Always
    name: metrics-server
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /tmp
      name: tmp-dir
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: metrics-server-token-nrdwl
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: kube-worker.t2mzj8lv
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: metrics-server
  serviceAccountName: metrics-server
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - emptyDir: {}
    name: tmp-dir
  - name: metrics-server-token-nrdwl
    secret:
      defaultMode: 420
      secretName: metrics-server-token-nrdwl
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2019-05-05T02:06:41Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2019-05-05T02:06:44Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2019-05-05T02:06:44Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2019-05-05T02:06:41Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://d49d30c912365bca3fb467f38a8e79a1c13c1f61d0a3edcc136d8ef6a5d7555e
    image: k8s.gcr.io/metrics-server-amd64:v0.3.1
    imageID: docker-pullable://k8s.gcr.io/metrics-server-amd64@sha256:78938f933822856f443e6827fe5b37d6cc2f74ae888ac8b33d06fdbe5f8c658b
    lastState: {}
    name: metrics-server
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: "2019-05-05T02:06:44Z"
  hostIP: 10.92.101.22
  phase: Running
  podIP: 10.36.0.5
  qosClass: BestEffort
  startTime: "2019-05-05T02:06:41Z"
tx19980520 commented 5 years ago

my yaml :

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: metrics-server
  namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: metrics-server
  namespace: kube-system
  labels:
    k8s-app: metrics-server
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  template:
    metadata:
      name: metrics-server
      labels:
        k8s-app: metrics-server
    spec:
      serviceAccountName: metrics-server
      volumes:
      # mount in tmp so we can safely use from-scratch images and/or read-only containers
      - name: tmp-dir
        emptyDir: {}
      containers:
      - name: metrics-server
        image: registry.cn-shanghai.aliyuncs.com/rancher/metrics-server:v0.3.3
        imagePullPolicy: Always
        command:
        - /metrics-server
        - --kubelet-insecure-tls
        - --kubelet-preferred-address-types=InternalIP
        volumeMounts:
        - name: tmp-dir
          mountPath: /tmp

It's same

E0730 01:53:08.101704       1 reststorage.go:147] unable to fetch pod metrics for pod istio-system/istio-pilot-58974b9b89-jx4vw: no metrics known for pod
E0730 01:53:08.105228       1 reststorage.go:147] unable to fetch pod metrics for pod istio-system/istio-egressgateway-5bfd96bbb8-8lq9q: no metrics known for pod
E0730 01:53:08.108672       1 reststorage.go:147] unable to fetch pod metrics for pod istio-system/istio-ingressgateway-64ccb6df5b-2jkz8: no metrics known for pod

I run top:

Error from server (NotFound): podmetrics.metrics.k8s.io "default/curl-test-774fdd546c-62gs4" not found

I have used the command mentioned in v0.3.1, but not work and who can tell me how to let pod export metrics, like springboot actuator?

ghost commented 5 years ago

same issue in EKS cluster with version 1.13

pabclsn commented 5 years ago

same here

tfluehmann commented 5 years ago

same here:

... discarding data: missing cpu usage metric, unable to get CPU for container "xyz" on node "1.1.1.1", discarding data: missing cpu usage metric]
kmdhali commented 4 years ago

did u get the workaround ? I have the same issue, nothing worked so far .."unable to fetch pod metrics for pod kube-system/kube-scheduler-ubuntu03: no metrics known for pod"

haimari commented 4 years ago

same here:

unable to fetch pod metrics for pod kube-system/kube-dns-64b69465b4-msj4m: no metrics known for pod
serathius commented 4 years ago

Looked through all logs provided in this issue. Looks like the problem is missing cpu metric in summary API exposed by Kubelet. This could be caused by two things:

Please provide me with:

William-Yi-Weng commented 4 years ago

Looked through all logs provided in this issue. Looks like the problem is missing cpu metric in summary API exposed by Kubelet. This could be caused by two things:

  • Fresh container (created < 15s ago), that cAdvisor didn't have chance to calculate CPU jet.
  • Kubelet failing to expose metric

Please provide me with:

  • K8s version
  • Kubernetes distro (EKS, GKE, kubeadm)
  • Name of pod that metric is missing
  • Results from running kubectl get --raw /api/v1/nodes/<node-name>/proxy/stats/summary with node that the pod is currently running

Hi @serathius , I meet the same issue. There are a lot of "userDefinedMetrics": null in my log.

Environment: Kubernetes version (use kubectl version): Kubernetes v1.12.10 Cloud provider or hardware configuration: AWS EC2 Debian

{
  "node": {
   "nodeName": "ip-172-20-32-10.ap-southeast-2.compute.internal",
   "systemContainers": [
    {
     "name": "kubelet",
     "startTime": "2019-11-12T10:23:39Z",
     "cpu": {
      "time": "2019-11-12T11:30:02Z",
      "usageNanoCores": 32655508,
      "usageCoreNanoSeconds": 118689813280
     },
     "memory": {
      "time": "2019-11-12T11:30:02Z",
      "usageBytes": 68579328,
      "workingSetBytes": 67903488,
      "rssBytes": 51367936,
      "pageFaults": 1006484,
      "majorPageFaults": 11
     },
     "userDefinedMetrics": null
    },
    {
     "name": "runtime",
     "startTime": "2019-11-12T10:23:34Z",
     "cpu": {
      "time": "2019-11-12T11:30:17Z",
      "usageNanoCores": 17944320,
      "usageCoreNanoSeconds": 100257936390
     },
     "memory": {
      "time": "2019-11-12T11:30:17Z",
      "usageBytes": 1503490048,
      "workingSetBytes": 402083840,
      "rssBytes": 103075840,
      "pageFaults": 605022,
      "majorPageFaults": 476
     },
     "userDefinedMetrics": null
    },
koy1619 commented 4 years ago

I had the same problem

E1113 05:09:57.112222       1 manager.go:111] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:k8s-node-1: unable to get CPU for container "paas-micro-voucherwallet" in pod default/paas-micro-voucherwallet-6b9796f9cf-lm6qv on node "10.127.0.17", discarding data: missing cpu usage metric

kubectl version v1.15.0

metrics-server-amd64:v0.3.6

protetore commented 4 years ago

We are facing the same issue with AKS.

E1203 16:28:46.464187       1 manager.go:111] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:aks-default-**********-*: unable to get CPU for container "*********" in pod default/*************** on node "*********, discarding data: missing cpu usage metric
taylorjacklespriggs commented 4 years ago

I have the same problem with EKS

E1212 18:34:52.078505       1 manager.go:111] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:ip-*****.us-west-2.compute.internal: [unable to get CPU for container "*****" in pod ***** on node "*****", discarding data: missing cpu usage metric, unable to get CPU for container "*****" in pod ***** on node "*****", discarding data: missing cpu usage metric]
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-20T04:51:35Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-eks-b8860f", GitCommit:"b8860f6c40640897e52c143f1b9f011a503d6e46", GitTreeState:"clean", BuildDate:"2019-11-25T00:55:38Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

metrics-server-amd64:v0.3.6

aneurinprice commented 4 years ago

I have run into this today. Bare metal cluster deployed though RKE:


E1218 09:37:23.124242       1 manager.go:111] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:susan: unable to get CPU for container "provision" in pod rook-ceph/rook-ceph-osd-prepare-susan-hbwsw on node "192.168.1.1", discarding data: missing cpu usage metric
mengjinmm commented 4 years ago
  • K8s版本

Hi @serathius ,

Kubernetes version (use kubectl version): Kubernetes v1.11.10

    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 237568,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 24
    }
   },
   {
    "podRef": {
     "name": "mpserver-9ff5c99cb-zbfw6",
     "namespace": "aps-os",
     "uid": "9b666408-2095-11ea-b11f-facfe37c9300"
    },
    "startTime": "2019-12-17T06:22:32Z",
    "containers": [
     {
      "name": "app-container",
      "startTime": "2019-12-17T06:22:34Z",
      "cpu": {
       "time": "2020-01-13T08:56:10Z",
       "usageNanoCores": 39026115,
       "usageCoreNanoSeconds": 64320370368232
      },
      "memory": {
       "time": "2020-01-13T08:56:10Z",
       "availableBytes": 1481117696,
       "usageBytes": 683126784,
       "workingSetBytes": 666365952,
       "rssBytes": 652050432,
       "pageFaults": 1091013,
       "majorPageFaults": 15
      },
      "rootfs": {
       "time": "2020-01-13T08:56:10Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 122880,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 13
      },
      "logs": {
       "time": "2020-01-13T08:56:10Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 1744044032,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:15Z",
     "usageNanoCores": 35137069,
     "usageCoreNanoSeconds": 64320797809107
    },
    "memory": {
     "time": "2020-01-13T08:56:15Z",
     "availableBytes": 1473691648,
     "usageBytes": 697933824,
     "workingSetBytes": 673792000,
     "rssBytes": 652099584,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:09Z",
     "name": "eth0",
     "rxBytes": 5905752000,
     "rxErrors": 0,
     "txBytes": 2904723456,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 5905752000,
       "rxErrors": 0,
       "txBytes": 2904723456,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2020-01-13T08:54:25Z",
      "availableBytes": 28321644544,
      "capacityBytes": 205110317056,
      "usedBytes": 107921408,
      "inodesFree": 55316094,
      "inodes": 55733632,
      "inodesUsed": 132,
      "name": "work-volume"
     },
     {
      "time": "2019-12-17T06:23:31Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "aps-sa-mpserver-token-779dj"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 1852088320,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 145
    }
   },
   {
    "podRef": {
     "name": "heron-85765687d-rkdnp",
     "namespace": "aps-os",
     "uid": "24b6185b-20cc-11ea-b11f-facfe37c9300"
    },
    "startTime": "2019-12-17T12:52:56Z",
    "containers": [
     {
      "name": "app-container",
      "startTime": "2019-12-17T12:53:00Z",
      "cpu": {
       "time": "2020-01-13T08:56:09Z",
       "usageNanoCores": 109393547,
       "usageCoreNanoSeconds": 224407213375890
      },
      "memory": {
       "time": "2020-01-13T08:56:09Z",
       "availableBytes": 1014657024,
       "usageBytes": 1152348160,
       "workingSetBytes": 1132826624,
       "rssBytes": 1112367104,
       "pageFaults": 1344407,
       "majorPageFaults": 22
      },
      "rootfs": {
       "time": "2020-01-13T08:56:09Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 122880,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 13
      },
      "logs": {
       "time": "2020-01-13T08:56:09Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 1377845248,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:06Z",
     "usageNanoCores": 115825183,
     "usageCoreNanoSeconds": 224408139077456
    },
    "memory": {
     "time": "2020-01-13T08:56:06Z",
     "availableBytes": 1009111040,
     "usageBytes": 1163407360,
     "workingSetBytes": 1138372608,
     "rssBytes": 1112408064,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:08Z",
     "name": "eth0",
     "rxBytes": 3610439010,
     "rxErrors": 0,
     "txBytes": 3615726300,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 3610439010,
       "rxErrors": 0,
       "txBytes": 3615726300,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2020-01-13T08:55:13Z",
      "availableBytes": 28316901376,
      "capacityBytes": 205110317056,
      "usedBytes": 93188096,
      "inodesFree": 55306830,
      "inodes": 55724368,
      "inodesUsed": 21,
      "name": "work-volume"
     },
     {
      "time": "2019-12-17T12:53:31Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "default-token-crjlt"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 1471156224,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 34
    }
   },
   {
    "podRef": {
     "name": "m-62da378b1ced488cbf6ae9bb2f52a363-55c45cf67f-dxdfs",
     "namespace": "aps-serving",
     "uid": "c6471d7f-35d6-11ea-8aac-facfe37c9300"
    },
    "startTime": "2020-01-13T07:31:56Z",
    "containers": [
     {
      "name": "m-62da378b1ced488cbf6ae9bb2f52a363",
      "startTime": "2020-01-13T07:32:00Z",
      "cpu": {
       "time": "2020-01-13T08:56:13Z",
       "usageNanoCores": 925953754,
       "usageCoreNanoSeconds": 3101067132539
      },
      "memory": {
       "time": "2020-01-13T08:56:13Z",
       "availableBytes": 865341440,
       "usageBytes": 208527360,
       "workingSetBytes": 208400384,
       "rssBytes": 208392192,
       "pageFaults": 88364,
       "majorPageFaults": 0
      },
      "rootfs": {
       "time": "2020-01-13T08:56:13Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 364544,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 58
      },
      "logs": {
       "time": "2020-01-13T08:56:13Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 2736128,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:14Z",
     "usageNanoCores": 969513944,
     "usageCoreNanoSeconds": 3102555056575
    },
    "memory": {
     "time": "2020-01-13T08:56:14Z",
     "availableBytes": 865296384,
     "usageBytes": 208572416,
     "workingSetBytes": 208445440,
     "rssBytes": 208437248,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:15Z",
     "name": "eth0",
     "rxBytes": 6904175,
     "rxErrors": 0,
     "txBytes": 1704438,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 6904175,
       "rxErrors": 0,
       "txBytes": 1704438,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2020-01-13T07:32:12Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "default-token-z97wx"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 3100672,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 58
    }
   },
   {
    "podRef": {
     "name": "m-62da378b1ced488cbf6ae9bb2f52a363-55c45cf67f-bnj9v",
     "namespace": "aps-serving",
     "uid": "d2a5444c-35d7-11ea-8aac-facfe37c9300"
    },
    "startTime": "2020-01-13T07:39:26Z",
    "containers": [
     {
      "name": "m-62da378b1ced488cbf6ae9bb2f52a363",
      "startTime": "2020-01-13T07:39:27Z",
      "cpu": {
       "time": "2020-01-13T08:56:04Z",
       "usageNanoCores": 904828295,
       "usageCoreNanoSeconds": 3102250219567
      },
      "memory": {
       "time": "2020-01-13T08:56:04Z",
       "availableBytes": 865374208,
       "usageBytes": 208494592,
       "workingSetBytes": 208367616,
       "rssBytes": 208334848,
       "pageFaults": 91152,
       "majorPageFaults": 0
      },
      "rootfs": {
       "time": "2020-01-13T08:56:04Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 364544,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 58
      },
      "logs": {
       "time": "2020-01-13T08:56:04Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 2752512,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:09Z",
     "usageNanoCores": 931714351,
     "usageCoreNanoSeconds": 3107182664348
    },
    "memory": {
     "time": "2020-01-13T08:56:09Z",
     "availableBytes": 865329152,
     "usageBytes": 208539648,
     "workingSetBytes": 208412672,
     "rssBytes": 208379904,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:15Z",
     "name": "eth0",
     "rxBytes": 7019156,
     "rxErrors": 0,
     "txBytes": 1733202,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 7019156,
       "rxErrors": 0,
       "txBytes": 1733202,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2020-01-13T07:40:12Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "default-token-z97wx"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 3117056,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 58
    }
   },
   {
    "podRef": {
     "name": "ambari-b757df688-9ntml",
     "namespace": "aps-os",
     "uid": "09b85417-166c-11ea-b11f-facfe37c9300"
    },
    "startTime": "2019-12-04T07:59:46Z",
    "containers": [
     {
      "name": "ambari-agent",
      "startTime": "2019-12-04T07:59:51Z",
      "cpu": {
       "time": "2020-01-13T08:55:59Z",
       "usageNanoCores": 0,
       "usageCoreNanoSeconds": 2285579670857
      },
      "memory": {
       "time": "2020-01-13T08:55:59Z",
       "availableBytes": 2118545408,
       "usageBytes": 35921920,
       "workingSetBytes": 28938240,
       "rssBytes": 21200896,
       "pageFaults": 184037,
       "majorPageFaults": 3
      },
      "rootfs": {
       "time": "2020-01-13T08:55:59Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 144138240,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 5482
      },
      "logs": {
       "time": "2020-01-13T08:55:59Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 36864,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     },
     {
      "name": "ambari-server",
      "startTime": "2019-12-04T07:59:50Z",
      "cpu": {
       "time": "2020-01-13T08:56:05Z",
       "usageNanoCores": 6779743,
       "usageCoreNanoSeconds": 17337106568245
      },
      "memory": {
       "time": "2020-01-13T08:56:05Z",
       "availableBytes": 1024729088,
       "usageBytes": 1131233280,
       "workingSetBytes": 1122754560,
       "rssBytes": 1114267648,
       "pageFaults": 1464035,
       "majorPageFaults": 15
      },
      "rootfs": {
       "time": "2020-01-13T08:56:05Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 865464320,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 7915
      },
      "logs": {
       "time": "2020-01-13T08:56:05Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 40960,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:11Z",
     "usageNanoCores": 5783017,
     "usageCoreNanoSeconds": 19623654827116
    },
    "memory": {
     "time": "2020-01-13T08:56:11Z",
     "availableBytes": 3143217152,
     "usageBytes": 1167212544,
     "workingSetBytes": 1151750144,
     "rssBytes": 1135517696,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:14Z",
     "name": "eth0",
     "rxBytes": 1052233320,
     "rxErrors": 0,
     "txBytes": 1933207227,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 1052233320,
       "rxErrors": 0,
       "txBytes": 1933207227,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2020-01-13T08:56:08Z",
      "availableBytes": 28316934144,
      "capacityBytes": 205110317056,
      "usedBytes": 8192,
      "inodesFree": 55306894,
      "inodes": 55724432,
      "inodesUsed": 3,
      "name": "work-volume"
     },
     {
      "time": "2019-12-04T08:00:18Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "default-token-crjlt"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 1009688576,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 13400
    }
   },
   {
    "podRef": {
     "name": "m-62da378b1ced488cbf6ae9bb2f52a363-55c45cf67f-rkhtw",
     "namespace": "aps-serving",
     "uid": "06aaecae-35e0-11ea-8aac-facfe37c9300"
    },
    "startTime": "2020-01-13T08:38:09Z",
    "containers": [
     {
      "name": "m-62da378b1ced488cbf6ae9bb2f52a363",
      "startTime": "2020-01-13T08:38:12Z",
      "cpu": {
       "time": "2020-01-13T08:55:54Z",
       "usageNanoCores": 0,
       "usageCoreNanoSeconds": 4059150765
      },
      "memory": {
       "time": "2020-01-13T08:55:54Z",
       "availableBytes": 934146048,
       "usageBytes": 139710464,
       "workingSetBytes": 139595776,
       "rssBytes": 139579392,
       "pageFaults": 70198,
       "majorPageFaults": 0
      },
      "rootfs": {
       "time": "2020-01-13T08:55:54Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 348160,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 55
      },
      "logs": {
       "time": "2020-01-13T08:55:54Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 32768,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:05Z",
     "usageNanoCores": 0,
     "usageCoreNanoSeconds": 5244332557
    },
    "memory": {
     "time": "2020-01-13T08:56:05Z",
     "availableBytes": 934100992,
     "usageBytes": 139755520,
     "workingSetBytes": 139640832,
     "rssBytes": 139624448,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:09Z",
     "name": "eth0",
     "rxBytes": 648,
     "rxErrors": 0,
     "txBytes": 0,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 648,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2020-01-13T08:38:12Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "default-token-z97wx"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 380928,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 55
    }
   },
   {
    "podRef": {
     "name": "postgres-exporter-prometheus-postgres-exporter-86cc5c5ccd-567vf",
     "namespace": "monitoring",
     "uid": "d06e9d58-e9ba-11e9-b11f-facfe37c9300"
    },
    "startTime": "2019-10-08T11:00:18Z",
    "containers": [
     {
      "name": "prometheus-postgres-exporter",
      "startTime": "2019-10-08T11:00:24Z",
      "cpu": {
       "time": "2020-01-13T08:56:11Z",
       "usageNanoCores": 2671750,
       "usageCoreNanoSeconds": 5181476415029
      },
      "memory": {
       "time": "2020-01-13T08:56:11Z",
       "availableBytes": 121233408,
       "usageBytes": 14893056,
       "workingSetBytes": 12984320,
       "rssBytes": 12328960,
       "pageFaults": 1047903,
       "majorPageFaults": 7
      },
      "rootfs": {
       "time": "2020-01-13T08:56:11Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 40960,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 8
      },
      "logs": {
       "time": "2020-01-13T08:56:11Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 28672,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:04Z",
     "usageNanoCores": 212804,
     "usageCoreNanoSeconds": 5181845191975
    },
    "memory": {
     "time": "2020-01-13T08:56:04Z",
     "availableBytes": 121192448,
     "usageBytes": 14934016,
     "workingSetBytes": 13025280,
     "rssBytes": 12369920,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:03Z",
     "name": "eth0",
     "rxBytes": 5623444413,
     "rxErrors": 0,
     "txBytes": 3406395930,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 5623444413,
       "rxErrors": 0,
       "txBytes": 3406395930,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2019-10-22T06:36:37Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "postgres-exporter-prometheus-postgres-exporter-token-2hcxh"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 69632,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 8
    }
   },
   {
    "podRef": {
     "name": "me-test-dep-79754cc579-x589k",
     "namespace": "aps-serving",
     "uid": "3c6a700f-2319-11ea-b11f-facfe37c9300"
    },
    "startTime": "2019-12-20T11:09:48Z",
    "containers": [
     {
      "name": "me-test",
      "startTime": "2019-12-20T11:09:49Z",
      "cpu": {
       "time": "2020-01-13T08:56:12Z",
       "usageNanoCores": 1399607,
       "usageCoreNanoSeconds": 2488115943927
      },
      "memory": {
       "time": "2020-01-13T08:56:12Z",
       "availableBytes": 1704710144,
       "usageBytes": 452972544,
       "workingSetBytes": 442773504,
       "rssBytes": 432578560,
       "pageFaults": 320466,
       "majorPageFaults": 1
      },
      "rootfs": {
       "time": "2020-01-13T08:56:12Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 197619712,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 24
      },
      "logs": {
       "time": "2020-01-13T08:56:12Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 979492864,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:03Z",
     "usageNanoCores": 1536814,
     "usageCoreNanoSeconds": 2488162716358
    },
    "memory": {
     "time": "2020-01-13T08:56:03Z",
     "availableBytes": 1704665088,
     "usageBytes": 453017600,
     "workingSetBytes": 442818560,
     "rssBytes": 432623616,
     "pageFaults": 0,
     "majorPageFaults": 0
    },
    "network": {
     "time": "2020-01-13T08:56:10Z",
     "name": "eth0",
     "rxBytes": 221281959,
     "rxErrors": 0,
     "txBytes": 226394555,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 221281959,
       "rxErrors": 0,
       "txBytes": 226394555,
       "txErrors": 0
      },
      {
       "name": "tunl0",
       "rxBytes": 0,
       "rxErrors": 0,
       "txBytes": 0,
       "txErrors": 0
      }
     ]
    },
    "volume": [
     {
      "time": "2019-12-20T11:10:34Z",
      "availableBytes": 33689927680,
      "capacityBytes": 33689939968,
      "usedBytes": 12288,
      "inodesFree": 8225074,
      "inodes": 8225083,
      "inodesUsed": 9,
      "name": "default-token-z97wx"
     }
    ],
    "ephemeral-storage": {
     "time": "2020-01-13T08:56:13Z",
     "availableBytes": 28316934144,
     "capacityBytes": 205110317056,
     "usedBytes": 1177112576,
     "inodesFree": 55306894,
     "inodes": 55724432,
     "inodesUsed": 24
    }
   },
   {
    "podRef": {
     "name": "hpatest-5585c489cf-sgwxv",
     "namespace": "aps-os",
     "uid": "acaef361-24a4-11ea-b11f-facfe37c9300"
    },
    "startTime": "2019-12-22T10:20:28Z",
    "containers": [
     {
      "name": "app-container",
      "startTime": "2019-12-22T10:20:29Z",
      "cpu": {
       "time": "2020-01-13T08:56:00Z",
       "usageNanoCores": 0,
       "usageCoreNanoSeconds": 580440040509
      },
      "memory": {
       "time": "2020-01-13T08:56:00Z",
       "availableBytes": 2144399360,
       "usageBytes": 7393280,
       "workingSetBytes": 3084288,
       "rssBytes": 1695744,
       "pageFaults": 12417,
       "majorPageFaults": 14
      },
      "rootfs": {
       "time": "2020-01-13T08:56:00Z",
       "availableBytes": 185939808256,
       "capacityBytes": 316928753664,
       "usedBytes": 131072,
       "inodesFree": 18123388,
       "inodes": 19660800,
       "inodesUsed": 18
      },
      "logs": {
       "time": "2020-01-13T08:56:00Z",
       "availableBytes": 28316934144,
       "capacityBytes": 205110317056,
       "usedBytes": 702345216,
       "inodesFree": 55306894,
       "inodes": 55724432,
       "inodesUsed": 417538
      },
      "userDefinedMetrics": null
     }
    ],
    "cpu": {
     "time": "2020-01-13T08:56:04Z",
     "usageNanoCores": 0,
     "usageCoreNanoSeconds": 580873889536
    },
    "memory": {
     "time": "2020-01-13T08:56:04Z",
     "availableBytes": 2144354304,
     "usageBytes": 7438336,
     "workingSetBytes": 3129344,
     "rssBytes": 1740800,
     "pageFaults": 0,
     "majorPageFaults": 0
    }
    "network": {
     "time": "2020-01-13T08:55:58Z",
     "name": "eth0",
     "rxBytes": 2184957590,
     "rxErrors": 0,
     "txBytes": 4749402967,
     "txErrors": 0,
     "interfaces": [
      {
       "name": "eth0",
       "rxBytes": 2184957590,
       "rxErrors": 0,
       "txBytes": 4749402967,
       "txErrors": 0
{
  "node": {
   "nodeName": "172.20.50.57",
   "systemContainers": [
    {
     "name": "kubelet",
     "startTime": "2019-10-22T06:35:37Z",
     "cpu": {
      "time": "2020-01-13T08:56:26Z",
      "usageNanoCores": 476436360,
      "usageCoreNanoSeconds": 2339460755836181
     },
     "memory": {
      "time": "2020-01-13T08:56:26Z",
      "usageBytes": 615063552,
      "workingSetBytes": 568414208,
      "rssBytes": 519962624,
      "pageFaults": 51769415536,
      "majorPageFaults": 678
     },
     "userDefinedMetrics": null
    },
    {
     "name": "runtime",
     "startTime": "2019-10-08T09:34:40Z",
     "cpu": {
      "time": "2020-01-13T08:56:18Z",
      "usageNanoCores": 1528928156,
      "usageCoreNanoSeconds": 5973077479991445
     },
     "memory": {
      "time": "2020-01-13T08:56:18Z",
      "usageBytes": 7351566336,
      "workingSetBytes": 4188823552,
      "rssBytes": 936706048,
      "pageFaults": 204554279824,
      "majorPageFaults": 1878
     },
serathius commented 4 years ago

Closing per Kubernetes issue triage policy

GitHub is not the right place for support requests. If you're looking for help, check Stack Overflow and the troubleshooting guide. You can also post your question on the Kubernetes Slack or the Discuss Kubernetes forum. If the matter is security related, please disclose it privately via https://kubernetes.io/security/.