flightlessmango / MangoHud

A Vulkan and OpenGL overlay for monitoring FPS, temperatures, CPU/GPU load and more. Discord: https://discordapp.com/invite/Gj5YmBb
MIT License
6.41k stars 286 forks source link

Trails of Cold Steel 3 crash again with recent mangohud git versions #743

Closed zany130 closed 2 years ago

zany130 commented 2 years ago

I started to get crashes in CSIII since 04/24/21 where the game would randomly crash after a few minutes to half an hour with

err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkSubmissionQueue: Command submission failed: VK_ERROR_DEVICE_LOST

in the proton log.

I found that disablinging mangohud fixed the crashes so I tried downgrading to a known working version https://github.com/flightlessmango/MangoHud/commit/4f9d870c724c4a6a97c223ac840c45de7fd9ba63

And I've been playing for several days without crashes on this version which I guess means there is a bug in a recent commit I have tested up to https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb

with no crashes so far

here is a link to a proton log of the crash steam-991270.tar.gz

Sys Info

inxi -b
System:
  Host: Kiseki Kernel: 5.17.4-256-tkg-pds arch: x86_64 bits: 64
    Desktop: KDE Plasma v: 5.24.4 Distro: Garuda Linux
Machine:
  Type: Desktop Mobo: ASRock model: X470 Taichi serial: <superuser required>
    UEFI: American Megatrends v: P4.86 date: 01/27/2022
CPU:
  Info: 6-core AMD Ryzen 5 2600X [MT MCP] speed (MHz): avg: 4169
    min/max: 2200/3600
Graphics:
  Device-1: NVIDIA GP104 [GeForce GTX 1080] driver: nvidia v: 510.60.02
  Display: x11 server: X.Org v: 21.1.3 with: Xwayland v: 22.1.1 driver: X:
    loaded: nvidia gpu: nvidia,nvidia-nvswitch resolution: 1: 2560x1080
    2: 3840x2160~60Hz
  OpenGL: renderer: NVIDIA GeForce GTX 1080/PCIe/SSE2
    v: 4.6.0 NVIDIA 510.60.02
Network:
  Device-1: Intel I211 Gigabit Network driver: igb
Drives:
  Local Storage: total: 2.96 TiB used: 3.02 TiB (102.2%)
Info:
  Processes: 408 Uptime: 5h 25m Memory: 31.27 GiB used: 12.91 GiB (41.3%)
  Shell: fish inxi: 3.3.15
zany130 commented 2 years ago

Can no longer reproduce on latest git

zany130 commented 2 years ago

Getting this issue again using https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb fixes the issue again here are the proton and dxvk log EDIT: github wasn't letting me upload a zip for some reason so I posted as a shared link with pcloud https://u.pcloud.link/publink/show?code=XZQYDpVZIVXjbyJ3BHhe9vxMhivgY0ksflB7

confirmed this does at least only happen on the latest mangohud versions by playing a full day (thanks to being stuck at home with covid, 😅 but mostly fine thankfully) and experiencingg several crashes vs playing on https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb and getting 0 crashes, config and everything else was the same between both tests (so the throttled config was disabled for both, enabling it however on the newer versions seem to cause mangohud to crash quicker in the same way with the same proton and dxvk errors)

jackun commented 2 years ago

If anything, current master should work better :D Have you trying upping "ulimits"?

zany130 commented 2 years ago

well I only tested unitl https://github.com/flightlessmango/MangoHud/commit/77bb7fd756820071bf1b135297bd81f719684170 right now Im doing a git bisect to see if I can narrow down exactly what commit between https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb and https://github.com/flightlessmango/MangoHud/commit/663bbd05a60c7d1e3fd352fdd8c55e96bd8af0f2 it may be ulimit related (Especially seeing as disabling esync help tremendously to the point where I couldn't reproduce it for a while.) also so I will try that after I find the commit the causes the issue (like I said https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb works for sure without any issues) I might as well post my ulimits here also

ulimit -a
Maximum size of core files created                           (kB, -c) unlimited
Maximum size of a process’s data segment                     (kB, -d) unlimited
Maximum size of files created by the shell                   (kB, -f) unlimited
Maximum size that may be locked into memory                  (kB, -l) unlimited
Maximum resident set size                                    (kB, -m) unlimited
Maximum number of open file descriptors                          (-n) 1048576
Maximum stack size                                           (kB, -s) 8192
Maximum amount of cpu time in seconds                   (seconds, -t) unlimited
Maximum number of processes available to a single user           (-u) 1048576
Maximum amount of virtual memory available to the shell      (kB, -v) unlimited

EDIT: tested https://github.com/flightlessmango/MangoHud/commit/6119fa80d132120b2143920318ce1437d65b71d9 to make sure it wasn't fixed in latest master and it crashed within 15 minutes (normally takes 30 to an hour)

crashes on https://github.com/flightlessmango/MangoHud/commit/f9cfdeb0804779a9957bcf956b9dfc63956a23b4

testing https://github.com/flightlessmango/MangoHud/commit/350dca5d2196c166d090fc783a6d8da607fe789e EDIT: crashes

testing https://github.com/flightlessmango/MangoHud/commit/e4b23c937eaf4daa44da6270d340e4434b1f23d1

zany130 commented 2 years ago

https://github.com/flightlessmango/MangoHud/commit/e4b23c937eaf4daa44da6270d340e4434b1f23d1 crashes so this is the first bad commit as the version before https://github.com/flightlessmango/MangoHud/commit/77bb7fd756820071bf1b135297bd81f719684170 EDIT: meant https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb ran all day yesterday without any issues

jackun commented 2 years ago

Wait, which one crashes? https://github.com/flightlessmango/MangoHud/commit/e4b23c937eaf4daa44da6270d340e4434b1f23d1 is older than 77bb7fd

zany130 commented 2 years ago

sorry meant https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb dosn't crash and https://github.com/flightlessmango/MangoHud/commit/e4b23c937eaf4daa44da6270d340e4434b1f23d1 crashes. copied and pasted the link without making sure it was the right one

zany130 commented 2 years ago

So tried reverting https://github.com/flightlessmango/MangoHud/commit/e4b23c937eaf4daa44da6270d340e4434b1f23d1 using GitHub desktop to see if that would help and it still crashes (pretty quickly too) so I'm guessing there's another commit that's causing the crash? Tried raising the file discreptor limit and it didn't seem to make a difference

So for now I'm just running https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb which isn't crashing

zany130 commented 2 years ago

crashed with https://github.com/flightlessmango/MangoHud/commit/5349226fa50f98c7d3328258112f48865b96cddb and https://github.com/flightlessmango/MangoHud/commit/4f9d870c724c4a6a97c223ac840c45de7fd9ba63 so not sure if the issue is even with mangohud.

on the one hand newer 6.7 version (and git) seem to crash consistently within 30 minutes. I have tried several times including recently to make sure it consistent and it is.

but old versions don't crash as often (or sometimes not at all) so while I did try this originally (but only for a day) I'm going to play without mangohud at all to see if that changes anything

zany130 commented 2 years ago

seem this is not a mangohud issue was able to reproduce without mangoud (although I had to leave the game running for about 5 hours vs with mangohud it crashes much sooner