Prometheus keeps series "alive" for 5 minutes after they get dropped from a scrape target. This leads to a slight overlap in the kube_node_labels metric if a node is relabelled.
This commit prioritises series having a node class label over series without.
This still does not work if a node class label changes from non-empty to non-empty but that should never happen. I don't think there is a generic way to handle this case.
Tested with LPG2 data at 2022-11-11 10:00:00Z.
Checklist
[x] Categorize the PR by setting a good title and adding one of the labels:
bug, enhancement, documentation, change, breaking, dependency
as they show up in the changelog
Prometheus keeps series "alive" for 5 minutes after they get dropped from a scrape target. This leads to a slight overlap in the
kube_node_labelsmetric if a node is relabelled.This commit prioritises series having a node class label over series without.
This still does not work if a node class label changes from non-empty to non-empty but that should never happen. I don't think there is a generic way to handle this case.
Tested with LPG2 data at
2022-11-11 10:00:00Z.Checklist
bug,enhancement,documentation,change,breaking,dependencyas they show up in the changelog