NetApp / harvest

Open-metrics endpoint for ONTAP and StorageGRID
https://netapp.github.io/harvest/latest
Apache License 2.0
146 stars 36 forks source link

Upgrade from 24.02 to 24.05.01 node throughput historical data missing #2967

Closed WayneShen2 closed 2 months ago

WayneShen2 commented 2 months ago

A note for the community

Problem

When we upgrade from 24.02 to 24.05.01 the node throughput historical data missing.

Configuration

No response

Poller

.

Version

24.05.01

Poller logs

No response

OS and platform

RHEL 8.9 docker

ONTAP or StorageGRID version

9.12.1P10

Additional Context

No response

References

No response

rahulguptajss commented 2 months ago

@WayneShen2 Could you add more details about this issue? For example, which metric is missing? What do you mean by missing historical data? Historical data is kept in Prometheus.

WayneShen2 commented 2 months ago

Seems like only Node Throughput data missing.

Untitled

rahulguptajss commented 2 months ago

Thanks for the context. Yes, earlier we were using the node_volume_read_data + node_volume_write_data query in the dashboard. However, to simplify the query, we have introduced a new metric called node_volume_total_data. This new metric is equivalent to the previous approach, but since older releases did not include it, you are missing historical data in Prometheus.

Could you try the query below for this panel? You should see historical data as well new data with this query:

sum(node_volume_read_data{datacenter=~"$Datacenter",cluster=~"$Cluster",node=~"$Node"}) by (node) + sum(node_volume_write_data{datacenter=~"$Datacenter",cluster=~"$Cluster",node=~"$Node"}) by(node)
WayneShen2 commented 2 months ago

Thank you, @rahulguptajss

That is working as expect

rahulguptajss commented 3 weeks ago

verified in 24.08