Prometheus queries return nothing

carlosedp / cluster-monitoring

Cluster monitoring stack for clusters based on Prometheus Operator

MIT License

740 stars 200 forks source link

Prometheus queries return nothing #89

Closed exArax closed 4 years ago

exArax commented 4 years ago

Hello,

This is the first time that happened to me, everything under the monitoring is in running state but when I query prometheus returns empty results. Grafana also shows empty results. What went wrong?

carlosedp commented 4 years ago

You need to check Prometheus targets if they are connected to the collectors.

exArax commented 4 years ago

How could I perform a check like this?

carlosedp commented 4 years ago

Go into Prometheus web GUI, Status -> Targets. They all should be in "UP" state.

Similar to this:

exArax commented 4 years ago

I made a make teardown. 192.168.1.2 is the IP of the master node 192.168.3 and 192.168.1.4 are worker nodes Again the prometheus IP is responding with an "Your connection was interruptedA network change was detected. ERR_NETWORK_CHANGED" error

Screenshot from 2020-08-20 20-16-17

carlosedp commented 4 years ago

Sorry, you need to debug what's wrong in the cluster not the stack. You should point the URL IP (in nip.io) to the IP your ingress controller runs.