search

cloudworkz / kube-eagle

A prometheus exporter created to provide a better overview of your resource allocation and utilization in a Kubernetes cluster.
MIT License
704 stars 46 forks source link

Metrics resource usage CPU and RAM all zeros #19

Closed mazzy89 closed 5 years ago

mazzy89 commented 5 years ago

My prometheus reports metrics like this for all the nodes equals to 0.

Any reasons?

eagle_node_resource_usage_cpu_cores{endpoint="http",instance="100.126.64.40:8080",job="kube-eagle",namespace="monitoring",node="ip-172-20-95-140.ec2.internal",pod="kube-eagle-d4c4bbf9f-4vgtx",service="kube-eagle"}
weeco commented 5 years ago

Hmm weird, what's your output of kubectl top nodes ?

mazzy89 commented 5 years ago 
➜ kubectl top nodes
error: metrics not available yet

maybe found the reason for that

weeco commented 5 years ago

Probably you are not running metrics-server in your cluster, which is a prerequisite for Kube Eagle to work. I should add that to the readme

mazzy89 commented 5 years ago

Hmm not really

it is deployed there but I got this erroro

metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843204       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-52-190.ec2.internal": no metrics known for node
metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843218       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-33-129.ec2.internal": no metrics known for node
metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843221       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-72-217.ec2.internal": no metrics known for node
metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843225       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-78-107.ec2.internal": no metrics known for node
metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843229       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-119-23.ec2.internal": no metrics known for node
metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843238       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-98-190.ec2.internal": no metrics known for node
metrics-server-5f64dbfb9d-9z9pd metrics-server E0406 09:41:28.843243       1 reststorage.go:129] unable to fetch node metrics for node "ip-172-20-63-138.ec2.internal": no metrics known for node
weeco commented 5 years ago

I am afraid I cannot be a great help figuring out why metrics-server cannot get metrics for your kubernetes nodes, but I'll leave this issue open for a while to see if someone else here experienced that problem as well and could help you.

mazzy89 commented 5 years ago

I'll figure out and report down here the result of my investigation

mazzy89 commented 5 years ago

Adding this arg should solve the issue

--kubelet-preferred-address-types=InternalIP

I'll test it later. Atm afk.

mazzy89 commented 5 years ago

Solved adding these two flags:

- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP

All good now 😎

weeco commented 5 years ago

Great, glad you could solve it