"ohm_gpuati_factor" doesn't report data with AMD driver 23.12.1

nickbabcock / OhmGraphite

Expose hardware sensor data to Graphite / InfluxDB / Prometheus / Postgres / Timescaledb

Other

426 stars 38 forks source link

"ohm_gpuati_factor" doesn't report data with AMD driver 23.12.1 #427

Closed Kybeer closed 9 months ago

Kybeer commented 11 months ago

I just updated my AMD driver from 23.11.1 to 23.12.1, and noticed the the ohm_gpuati_factor FPS metric is no longer repoting anything.

It doesn't appear at all if I check my Prometheus endpoint http://localhost:4445/metrics. I don't know if anything can be done on OhmGraphite side, or if it's a driver issue. I'm on v0.31.0 (latest right now).

All other metrics looks fine.

nickbabcock commented 10 months ago

Restarting the computer may be worth trying if you haven't already.
I'd be curious if you see the sensor listed in the GUI https://github.com/LibreHardwareMonitor/LibreHardwareMonitor#where-can-i-download-it (if not, we're looking at an upstream issue).

Kybeer commented 10 months ago

Restarting the computer may be worth trying if you haven't already.

I'd be curious if you see the sensor listed in the GUI https://github.com/LibreHardwareMonitor/LibreHardwareMonitor#where-can-i-download-it (if not, we're looking at an upstream issue).

Restarting makes no difference.

The value is visible in LibreHardwareMontor still:

But no metric in the Prometheus endpoint anymore for ohm_gpuati_factor{sensor="Fullscreen FPS"}

nickbabcock commented 10 months ago

Thank you for checking. The plot thickens. Can you try running OhmGraphite interactively (in an adminstrator terminal)?

./OhmGraphite stop
./OhmGraphite run

A few have reported that doing this can yield more metrics and I'm curious if this is one of those cases.

Kybeer commented 10 months ago

Thank you for checking. The plot thickens. Can you try running OhmGraphite interactively (in an adminstrator terminal)?
./OhmGraphite stop
./OhmGraphite run
A few have reported that doing this can yield more metrics and I'm curious if this is one of those cases.

Thanks for your time on this. I did what you suggested but it makes no difference. The terminal shows a log of metrics and the fps is still always 0 (line 69), just like the Prometheus endpoint. I attached the terminal output as text here.

ohm.txt

Since this happened after a driver update perhaps its reporting FPS in a new way?

Still ok here though:

nickbabcock commented 10 months ago

To clarify, is the fps being reported as zero or not reported at all?

The sensor is being detected so that is good. My hunch is that if you were to flip on debug logging https://github.com/nickbabcock/OhmGraphite#debugging-tips there may be more log lines related to the faulty sensor.

Kybeer commented 10 months ago

To clarify, is the fps being reported as zero or not reported at all?

The sensor is being detected so that is good. My hunch is that if you were to flip on debug logging https://github.com/nickbabcock/OhmGraphite#debugging-tips there may be more log lines related to the faulty sensor.

Actually, ohm_gpuati_factor{sensor="Fullscreen FPS"} doesnt' return any data at all. The metric is not even appearing in the Prometheus endpoint (http://localhost:4445/metrics)

nickbabcock commented 10 months ago

In the log posted earlier, I saw the following snippet:

Sensor added: /gpu-amd/0/factor/0 "Fullscreen FPS"

When you switched to debug logging, were there more lines related to /gpu-amd/0/factor/0?

Kybeer commented 10 months ago

OhmGraphite.log

When you switched to debug logging, were there more lines related to /gpu-amd/0/factor/0?

I enabled Debug but I don't think there's much difference. Attaching the log file here.

nickbabcock commented 10 months ago

The mystery deepens! I see /gpu-amd/0/factor/0 listed in the first log but not in the second. What do you think changed?

Kybeer commented 10 months ago

The mystery deepens! I see /gpu-amd/0/factor/0 listed in the first log but not in the second. What do you think changed?

I haven't changed anything myself since the driver update three weeks ago. There's been Windows updates in that time though..

nickbabcock commented 10 months ago

Still scratching my head a bit. Thank you for your patience

Try the OhmGraphite artifact from the PR that bumps to LibreHardwareMonitor 0.9.3

Probably also worth double checking the official release of LibreHardwareMonitor 0.9.3 still shows everything correctly too.

Kybeer commented 9 months ago

Still scratching my head a bit. Thank you for your patience

Hi again. Driver 24.1.1 dropped today and FPS is now working again :)

nickbabcock commented 9 months ago

That's great news, I'll go ahead and close this issue out.