search

dotdc / grafana-dashboards-kubernetes

A set of modern Grafana dashboards for Kubernetes.
Apache License 2.0
2.63k stars 368 forks source link

[bug] node dashboard only shows latest instance #102

Closed uhthomas closed 6 months ago

uhthomas commented 8 months ago

Describe the bug

Some panels are using node to filter, and others are using a hidden instance variable (label_values(node_uname_info{nodename=~"(?i:($node))"}, instance)). If a node changes its IP, then some panels will look normal and others will be missing data.

image

How to reproduce?

  1. Collect node metrics.
  2. Change IP of node.
  3. Observe the node dashboard has some unaffected panels, and others which only show the latest 'instance'.

Expected behavior

It should probably show all instances of a node.

Additional context

No response

dotdc commented 7 months ago

Hi @uhthomas,

I've been thinking about this and don't see a fix for this one. The node / instance hack was used to have convenient names while using the dashboard, I'd like to keep it that way.

For me, a node changing IP is a rare corner case, but I guess it depends on your setup.

How often do you have IP changes on your nodes?

dotdc commented 6 months ago

Documented in 34a469f