search

GPUOpen-Tools / radeon_gpu_profiler

Radeon GPU Profiler (RGP) is a tool from AMD that allows for deep inspection of GPU workloads.
396 stars 41 forks source link

Incorrectly reported occupancy on GCN hardware #53

Closed PMunkes closed 3 years ago

PMunkes commented 3 years ago

I recently profiled Devil May Cry 5 on an RX 480 and an RX 6800 on 21.3.1 and noticed that the occupancy is most likely reported incorrectly on the RX 480. Specifically the occupancy values seem to be reported as 1/4 of the true value:

Scene: grafik

Profile on RX 480: grafik

Profile on RX 6800: grafik

I have noticed this in several settings and also on a Vega 56, so this is not specific to DMC5 or Polaris.

chesik-amd commented 3 years ago

Thanks for reporting this. We will see if we can reproduce this. In the meantime, would you be able to share the .rgp file (from either the RX 470 or Vega 56) where you see this?

Thanks, Chris

chesik-amd commented 3 years ago

One other thing to check -- if you disable Instruction timing when capturing the profile (in Radeon Developer Panel), does it change the behavior?

chesik-amd commented 3 years ago

Hi again. Did you recapture the profile on the RX 480 after installing 21.3.1? I can reproduce the issue on Polaris with an older driver (like 21.2.3), but not with 21.3.1.

There was an issue in the RGP support in the driver that would cause this exact issue when instruction tracing was enabled during capture. The issue was fixed in 21.3.1, and using that driver I no longer see this issue on Polaris. I'm hoping to be able to confirm that what you're seeing is the same issue that has already been fixed, rather than a different issue that we are not aware of.

Thanks, Chris

PMunkes commented 3 years ago

Unfortunately I don't currently have access to the Vega56 until the weekend and I no longer have my RX480. I do have an HD7970 that I would try this on. I have uploaded the RGP file for the RX480 here: https://uni-muenster.sciebo.de/s/gAsV8lvpiMdML6k

PMunkes commented 3 years ago

I only recorded the trace on the 21.2.3 driver, but didn't get to report this until yesterday.

chesik-amd commented 3 years ago

OK, thanks. Looking at your .rgp file, it appears the driver bug I referred to is what is happening. Basically, we are missing the wavefront data from additional shader engines when instruction tracing has been enabled.

Here's a screenshot , showing there is only data from Shader Engine 0 (the Wavefront Occupancy view is set to "color by shader engine"):

image

You can confirm this by recapturing without Instruction tracing enabled on 21.2.3. Or you can try the 21.3.1 driver, which has the fix for the driver issue.

Again, thanks for bringing this up!

PMunkes commented 3 years ago

Thanks, good to know that this was already fixed. Also, there really is a lot more to this tool than I was aware of.