Closed candlerb closed 3 years ago
Thanks for pointing the issue, it's is fixed now.
I just checked the latest install of 1860 (node-exporter-full_rev23.json
), and the panel Storage Disk > Instantaneous Queue Size
still shows:
rate(node_disk_io_now{instance="$node",job="$job"}[$__rate_interval])
I believe it should be instead:
node_disk_io_now{instance="$node",job="$job"}
In addition:
I/O ops/sec (iops)
; I think this should be Misc > none
. The metric is the queue depth, so it's "number of outstanding requests", which is dimensionless and definitely not a rate of any kind.The corrected panel would look like this:
This shows the number of active requests is either 0 or 1 at each sample time, which is correct.
(Aside: I can prepare a PR for this, but I have other graphs I want to check first)
Since I'm running Grafana 8.0.5, and the dashboard is for 7.3.7, the diffs turned out to be rather large, especially because large chunks of indentation have changed due to a change in panels/targets layout.
Probably best just to apply the two changes by hand:
- "expr": "rate(node_disk_io_now{instance=\"$node\",job=\"$job\"}[$__rate_interval])",
+ "expr": "node_disk_io_now{instance=\"$node\",job=\"$job\"}",
...
- "format": "iops",
+ "format": "none",
It was a regression on d52e0ec13cbd5bc916eedf6170f1ee99e914d889 Let me fix and check the other changes that you reported over the current dashboard.
Thanks Brian,
In the panel "Disk Detail" is a graph "Disk IOs Current In Progress". The query it uses is:
However, node_disk_io_now is not a counter, it's a gauge:
The
io_now
value is field 9 documented here:Therefore I believe the
irate(...)
wrapper needs to be removed. This is already an instantaneous snapshot of the outstanding I/O requests.