search

hjacobs / kube-ops-view

Kubernetes Operational View - read-only system dashboard for multiple K8s clusters
https://kubernetes-operational-view.readthedocs.io/
GNU General Public License v3.0
1.82k stars 256 forks source link

Unable to fetch metrics #206

Open Patafix opened 5 years ago

Patafix commented 5 years ago

Metrics server is running fine with no warnings, installed with helm on k8s 1.11

hostNetwork:
    enable: true
args:
  - --logtostderr
  - --kubelet-insecure-tls=true
  - --kubelet-preferred-address-types=InternalIP
image:
  tag: v0.3.1

As soon as i launch kube-ops-view i get a lot of error on the metrics servers, full of this kind of error (no other errors)

E0227 18:49:06.479377 1 reststorage.go:144] unable to fetch pod metrics for pod bar/foo: no metrics known for pod

Kube Ops View is installed with helm : 

image:
  repository: hjacobs/kube-ops-view
  tag: 0.9.1
rbac:
  create: false
  serviceAccountName: kube-ops-view

Logs of kubeopsview : 

INFO:kube_ops_view.main:Listening on :8080..
INFO:kube_ops_view.kubernetes:Heapster node metrics not available (yet)
INFO:kube_ops_view.kubernetes:Heapster pod metrics not available (yet)
INFO:kube_ops_view.update:Discovered new cluster XXXXX:443 (https://XXXX:443).
100.96.9.1 - - [2019-02-27 18:45:46] "GET /health HTTP/1.1" 200 117 0.000602
100.96.9.1 - - [2019-02-27 18:45:47] "GET /health HTTP/1.1" 200 117 0.000468
INFO:kube_ops_view.kubernetes:Heapster node metrics not available (yet)
INFO:kube_ops_view.kubernetes:Heapster pod metrics not available (yet)
INFO:kube_ops_view.kubernetes:Heapster node metrics not available (yet)
INFO:kube_ops_view.kubernetes:Heapster pod metrics not available (yet)
100.96.9.1 - - [2019-02-27 18:45:56] "GET /health HTTP/1.1" 200 117 0.000484
100.96.9.1 - - [2019-02-27 18:45:57] "GET /health HTTP/1.1" 200 117 0.000478

No errors in the logs

hjacobs commented 5 years ago

Is kubectl top pod working for you? (that uses the same endpoint)

Patafix commented 5 years ago

@hjacobs Yes it's work, even kube-ops-view work fine, but generate a lot of errors in the metrics pod.

The more i look it's seem kube-ops-view request terminated pod (Cron Job)

bobhenkel commented 5 years ago

Thanks this helped me with my local workstation cluster I'm running via kind cluster getting metrics server to consume metrics in addition to allowing kube ops view to show usage values.

In case this helps someone else I added this to the metrics server deployment.(Not sure you need the - /metrics-server but I'm using it and it works)

      containers:
      - command:
        - /metrics-server
        - --logtostderr
        - --kubelet-preferred-address-types=InternalIP
        - --kubelet-insecure-tls
        - --metric-resolution=10s
        image: k8s.gcr.io/metrics-server-amd64:v0.3.3