intel / pcm

Intel® Performance Counter Monitor (Intel® PCM)
BSD 3-Clause "New" or "Revised" License
2.82k stars 476 forks source link

Grafana Shows Wrong Core_Frequency Value, But Data is Right #789

Open alastorid opened 4 months ago

alastorid commented 4 months ago

The system is a DL560G8 with 4x4657LV2. Here's what the Grafana looks like:

image

The metric 7.86e-7 for CPU frequency seems incorrect, as I expect the CPUs to run between 1 to 2.9 GHz, or possibly at 0.8 GHz. I've monitored the dashboard for a while and noticed that the frequency never reaches 2.x GHz, even when the target system is fully loaded.

I checked the data from back-end and the result seems good:

image

Attached are some metric files from the system, along with the prometheus.json dashboard file. Not all data on Grafana are incorrect: for example, DDR read/write and cache hit rates appear correct. I've checked the dashboard file and can't find an obvious reason for the unexpected Core_Frequency value. Any pointers would be appreciated! Dashbaord

Metrics

P.S. The Grafana instance is running in Docker on a freshly installed Red Hat Enterprise Linux 9.4, launched using the script ./start-prometheus.sh target.txt

rdementi commented 4 months ago

thanks for the report. This looks like a bug to me..

rdementi commented 4 months ago

this seems to be specific to the way how this pcm data is exposed in prometheus format. The PCM start.sh grafana script (telegraf/inflixdb/grafana stack) shows these metrics correctly.

rdementi commented 4 months ago

I think I have root-caused the issue.

alastorid commented 4 months ago

Glad to hear it’s been figured out! Looking forward to the update.