doitsujin / dxvk

Vulkan-based implementation of D3D8, 9, 10 and 11 for Linux / Wine
zlib License
13.18k stars 849 forks source link

AMD RADV driver discussion #252

Closed doitsujin closed 4 years ago

doitsujin commented 6 years ago

Some games may lock up the GPU when using the RADV Vulkan driver on AMD cards, which results in a frozen system. Unless this is caused by an obvious DXVK bug (i.e. there are Vulkan validation errors when VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_standard_validation is set), please do not open a new issue if you encounter one of these hangs.

Instead, please comment on this thread and:

Creating a hang report

In order to obtain a hang report from RADV for a specific game, set the RADV_DEBUG environment variable and redirect stderr and stdout to a file as follows:

export RADV_TRACE_FILE=/***/radv-trace.txt
export RADV_DEBUG=allbos,syncshaders,vmfaults
export WINEDEBUG=-all
wine game.exe 2>&1 | tee hang_report.txt

For games which launch themselves through Steam, modifying the launch options may be necessary.

Important: Please make sure that you have spirv-tools installed and that the spirv-dis executable is in your PATH.

sierkov-bot commented 6 years ago

Witcher 3

GPU: RX 560 4GB (mesa-git , llvm-svn, dxvk-git 20180410.1fb22a6)

Settings: High presets, disabled vsync.

How to reproduce: In the beginning of the game, after Geralt wakes up from the dream and they are about to hit the road they smell ghouls. When either me or Vesemir hits the ghoul GPU hangs, not at the exact moment of hit, but soon after it. I've been able to reproduce it several times, every time my GPU hangs when i fight the ghouls.

Logs hang report d3d11_log dxgi_log

My saves and in-game settings: The Witcher 3.tar.gz

shmerl commented 6 years ago

@rserkov : to avoid this in the log:

sh: umr: command not found

You can build umr debugger from here.

sierkov-bot commented 6 years ago

@shmerl i just skimmed through the log since i don't understand much of what it says. Should i redo hang report with umr installed?

doitsujin commented 6 years ago

@rserkov Not sure if umr provides all that much useful information, although it wouldn't hurt. Looks like you don't have spirv-tools installed though, getting the SPIR-V disassembly would be rather important to see if there is maybe something wrong with the shaders.

sierkov-bot commented 6 years ago

@doitsujin redid with spirv-tools and umr installed, hang report. Please let me know if there is anything else i can do to help.

jarrard commented 6 years ago

Is it really a hang or a screen freeze, can you type blind into a console login and password then reboot (ie. ctrl+f1).

I have encounter a system hand/freeze with my 1080ti but discovered it was not a hang but a unrecoverable screen freeze that which you can still type during and reboot via a terminal console.

sr-tream commented 6 years ago

@jarrard on AMD, this freeze of the screen after a while goes into complete suspension of the system.

Kzimir commented 6 years ago

Assassin's Creed III

GPU: RX 560 4GB (mesa-git , llvm-5.0.1, dxvk-r952.adb1789). Same issue with llvm-git.

Settings: Normal Settings, VSync disabled

How to reproduce: I'm in Boston and if i enable the Eagle Vision, the game crash and the system hangs. Need to hard reboot. The system can hangs after to play for a long time.

Logs hang report d3d11_log dxgi_log

Nerellus commented 6 years ago

Star Trek Online

GPU: RX 570 8GB mesa: git @ 6a519a157b5fe5d449444c04a0429e8a24546e9c llvm: svn @ 330092 (commit 319534 reverted) dxvk: git @ 31ed6e5cd34a9b3fb46d19f975f2ba21e56493be

Settings: Defaults

How to reproduce: cd /path/to/Star\ Trek\ Online_en/Star\ Trek\ Online/Live wine x64/GameClient.exe -Locale English -server 208.95.186.11 GPU hangs while loading login screen

Logs: hang_report.txt radv-trace.txt GameClient_d3d11.log GameClient_dxgi.log

apitrace: STO.dxvk.trace STO.win7.trace

Unfortunately I can't get a trace with wined3d. This trace was made with dxvk+amdvlk (which does not hang here), when replayed with RADV it hangs as normal. Added additional apitrace from Windows 7.

mradermaxlol commented 6 years ago

Okay, straight outta https://github.com/doitsujin/dxvk/issues/193, eh? :)

Here's the hang report: hang_report.txt I ran as mentioned in the how-to (I ran TheCrew.exe from UPlay's game directory), with spirv-tools installed. I compiled {lib32-,}llvm-svn_r330096 with that amdgpu thing reverted & {lib32-,}mesa-git_101626.6a519a157b. The only visual change was that with all that RADV debugging enabled I could see the chat thingie rendering, though everything else remained the same - there's a static image, background sounds and that's it. Here's the output of running the game with only DXVK_DEBUG_LAYERS=1 set: consolelog.txt

DXVK version used: https://github.com/doitsujin/dxvk/commit/98b8d410168e526dba6fe1950df111a631e6a8de

GloriousEggroll commented 6 years ago

Overwatch hangs on llvm 6.0.0, 5.0/5.0.1/5.0.2 can be used

tdjb commented 6 years ago

Confirming Overwatch hangs on llvm 6.0.0 as well as on llvm 7.0.0-svn with mesa-git.

stalkerg commented 6 years ago

Maybe will be better if we make also issues on mesa and llvm bug trackers and put links here? In my opinion, if GPU hangs it driver problem.

exolyte commented 6 years ago

Event[0]

The game hangs in the first loading screen after the intro.

mesa: 18.1 (96ed371) llvm: 7.0 (331148) dxvk: 4c298d4 GPU: RX 570

event0_d3d11.log event0_dxgi.log event0-hang_report.txt event0-radv-trace.txt Apitrace

EDIT 7th of june: Event[0] still hangs with the hellblade mesa workaround. I've added an apitrace to reproduce the hang.

asumagic commented 6 years ago

Overwatch Seems easier to reproduce the hang with graphics set to absolute maximum when using RADV_DEBUG. Happens on low settings as well.

GPU: RX580

Hang report Nothing worth mentioning in _d3d11 and _dxgi logs, but here they are anyway.

jarrard commented 6 years ago

You guys having hangs should monitor your GPU temperatures while playing with either a overlay or a log to txt method. I believe some radeon cards will start to crash above 85c

shmerl commented 6 years ago

Sapphire cards are cooled pretty well, they never reach such high temperature for me, even on 100% load (and I do monitor it, you can run something like Ksysguard in parallel, it has neat hardware monitor features where you can add any sensor to show a dynamic graph). But I didn't have GPU hangs either so far with dxvk.

Is there a way to test a hang with TW3? I can try some save and check if it's a temperature issue or not.

shmerl commented 6 years ago

Example (99% GPU load with dxvk / The Witcher 3, 1920x1200 Sapphire Pulse Vega 56):

ksysguard_tw3_dxvk

It maxes out around at 74°C for me.

jarrard commented 6 years ago

Yeah looks ok, you can also run 3dmark on max for 3-4 runs to ensure its solid. Assuming the latest 3dmark stresses the GPU's enough.

shmerl commented 6 years ago

I think cooling is OK. Would be interesting to confirm if hangs are not cooling related.

doitsujin commented 6 years ago

@jarrard Please don't spread the idea that any of this is caused by overheating GPUs. That's complete nonsense. I opened this meta-issue because I know for a fact that these problems are reproducible and are generally caused by either LLVM bugs or sometimes DXVK bugs.

asumagic commented 6 years ago

I've been playing TW3 for hours without a hang (on the first released version - not the up to date one, because I haven't updated it yet and TW3 is no-DRM so I didn't bother) - it didn't really get over 75°C and that's not a problem. The hang happens reproducibly on OW on low, capped fps, so it's definitively not an issue.

sr-tream commented 6 years ago

The witcher 3 The more FPS in the game, the less likely the system will hang. with RADV_DEBUG I get 1 fps and the game does not hang at all. When FPS 60 everything hangs from one dog bite

I can't attach gpu hang, becaus system is not freezed with low fps

asumagic commented 6 years ago

@sr-tream I noticed the same thing with OW. Try to bump your settings to maximum and somehow making it render to 4k or 8k or whatever to maximize gpu usage, I guess.

shmerl commented 6 years ago

@doitsujin:

I know for a fact that these problems are reproducible and are generally caused by either LLVM bugs or sometimes DXVK bugs.

Are those bugs reported to llvm? I.e. is there a chance they'll be fixed in next release?

sr-tream commented 6 years ago

@AsuMagic with 13fps hang is present. Hanging is only visible during fights

doitsujin commented 6 years ago

Are those bugs reported to llvm?

I cannot report bugs to LLVM directly. I can only report issues to some of the RADV developers and hope they eventually figure out what's wrong on the LLVM side of things.

asumagic commented 6 years ago

Overwatch again, maximum graphics (still hangs on low), RX580, this time with a proper radv-trace:

Kerrung commented 6 years ago

@doitsujin Using DXVK, the game hasn't any rendering distortions, but after a few seconds of playing the game, Assassin's Creed Unity totally crashes/hangs up my system. After that, the only thing I can do is hard reboot. Unfortunately, I absolutely could not write down the apitrace using DXVK because here does not appear the *.trace file. I assume that this is due to the strict binding of all games of Ubisoft to Uplay.

Using WineD3D, the game has heavy rendering distortions because of which I can hardly see anything in the game, and after a few seconds of playing the game crashes. Unfortunately, I absolutely could not write down the apitrace using WineD3D because here does not appear the *.trace file.

In addition, I tried to write down the apitrace on the MS Windows, but there does not appear the *.trace file too.

Software information

Assassin's Creed Unity, minimal graphics settings used.

System information

Apitrace file(s)

Log files

Terminal output

BlauerHunger commented 6 years ago

Which bug reports on the mesa or llvm bugtracker are related to this?

shmerl commented 6 years ago

It can be on llvm bug tracker.

alexzzd commented 6 years ago

Witcher3 don't hanging anymore with DXVK 0.53 and DXVK_USE_PIPECOMPILER=1 (sorry for bad english)

ghost commented 6 years ago

@alexzzd just so you know: you don't have to write "sorry for bad english" in your messages, it's fine - a LOT of english-speaking users on the internet are not native english speakers :)

doitsujin commented 6 years ago

Frostpunk doesn't hang anymore either so it seems like there have been some bugfixes recently on the LLVM side of things. Then again, I never experienced a single hang in The Witcher 3.

@abba Far Cry 5 being red is an unrelated bug that also affects Nvidia, but only happens under extremely weird conditions, where just moving code around can either fix or trigger the issue when DXVK is compiled with certain compilers. @ZeroFault tried to help debug it but as of right now, we don't understand this issue at all.

shmerl commented 6 years ago

I've never had hangs in TW3 too for the reference. But I haven't played it extensively besides a few tests here and there.

exolyte commented 6 years ago

Final Fantasy XIV

The game hangs when loading into the game, only when the real-time reflections setting is on.

mesa-git: b9fb2c266a llvm-svn: 333555 dxvk: 621aed5 gpu: RX 570 apitrace: https://mega.nz/#!vMthmATI!q8wARC8A9cv6TDmk4iyF4CMPwClTDLuSF9mtpMF2J_k

ffxiv_d3dretrace_d3d11.log ffxiv_d3dretrace_dxgi.log ffxiv_hang_report.txt ffxiv_radv_trace.txt

hakzsam commented 6 years ago

Can you guys try this patch https://patchwork.freedesktop.org/patch/226715/ ?

It fixes a GPU hang with "Seven: The Days Long Gone", at least. Note that it doesn't fix the GPU hang with Hellblade (but I have something locally that helps, not quite ready yet).

Thanks!

hakzsam commented 6 years ago

Someone reported me that "Assassin's Creed III" is also fixed with that patch.

Kerrung commented 6 years ago

@hakzsam

Can you guys try this patch https://patchwork.freedesktop.org/patch/226715/ ? It fixes a GPU hang with "Seven: The Days Long Gone", at least. Note that it doesn't fix the GPU hang > with Hellblade (but I have something locally that helps, not quite ready yet). Thanks!

I just tried your patch and it totally fixes this GPU hang it Assassin's Creed Unity! Thank you and @doitsujin very much! My issue is now closed!

Nerellus commented 6 years ago

@hakzsam fixes the Star Trek Online hang as well. Thanks. :smiley:

beniwtv commented 6 years ago

Unfortunately, Star Trek Online still hangs the GPU for me with the patch :( @portentum What settings do you use for STO? Also what GPU do you have?

ghost commented 6 years ago

@hakzsam GTA V hang is fixed ! (Tested with LLVM 6.0.1, LLVM 7 still hangs)

Nerellus commented 6 years ago

@beniwtv I'm using a RX 570 8GB. Settings album / Gameprefs.Pref

wine-staging git 8df70b8 + vulkan 1.1 patches [1] [2] llvm 7 svn r333673 + hakzsam's patch mesa git f00fcfb + doitsujin and hakzsam's patches [1] [2] dxvk git 9ff17b0

update: I dropped all of the patches I mention above except hakzsam's fix for the hang and the game still works. So you can disregard the wine, llvm patches, and doitsujin's mesa patch.

jarrard commented 6 years ago

Wish us NVIDIA users could get a hang fix sorted out for that KCD tavern crash bug :(

hakzsam commented 6 years ago

@beniwtv what GPU?

beniwtv commented 6 years ago

@hakzsam RX 480 8GB reference card, using Mesa-Git from yesterday with only your patch. LLVM 6.0.0. I was thinking it might be that LLVM version. Should I try with LLVM-git?

Kerrung commented 6 years ago

@hakzsam thank you so much for your patch! It is realy works great for me! But when it will be upstreamed? Regards.

Kerrung commented 6 years ago

@spinozaure

GTA V hang is fixed ! (Tested with LLVM 6.0.1, LLVM 7 still hangs)

But I haven't GPU hangs with this patch and with LLVM 7. screenshot from 2018-06-03 02-42-14

mradermaxlol commented 6 years ago

Tested GTA V (Steam) with Mesa 18.1.1 (+ patch from this thread) & LLVM 6.0.0 & dxvk @ https://github.com/doitsujin/dxvk/commit/217399926d1c44d8c2532de62579bf9b23fa9adc on my R7 370 (amdgpu driver), works good. There's some incorrect rendering, though, looks like the shadows are messed up.

lodriguez commented 6 years ago

@mradermaxlol same here and when I set shader quality to high or very high, the game crashes as soon as game play starts.