VR applications fail to load OpenXR library with stack overflow in Proton 8.0+

[peterfab9845]

Compatibility Report

• Name of the game with compatibility issues: Beat Saber, SCP: Secret Laboratory, VRChat, vrcmd.exe --spamhaptics from Windows SteamVR install (every non-native VR application I tried)
• Steam AppID of the game: 620980, 700330, 438100, 250820

System Information

• GPU: RTX 3060 Ti
• Video driver version: nvidia 535.113.01
• Kernel version: 6.5.8
• Link to full system information report as Gist: https://gist.github.com/peterfab9845/5daa6a62e8e8af90f7c0940c6f096b82
• SteamVR system report: SteamVR-2023-11-02-AM_04_30_55.txt.gz
• Proton version: 1697709592 experimental-8.0-20231019, also observed identical results with Proton 8.0-4
• Steam client version: 1698890449 (Steam Client Beta), also observed with a non-beta client
• SteamVR version: beta 2.0.10, also observed with 1.27.5, 2.0.8, and 2.0.9 (all that were tried)

I confirm:

• [x] that I haven't found an existing compatibility report for this game.
• [x] that I have checked whether there are updates for my system available.

Symptoms

1. Game appears to start in Steam and shows as "Next Up" in SteamVR, but nothing ever appears and the game remains as "running" until killed.
2. SteamVR wakes up for a few seconds upon launching the game, then goes back into standby. (Same results if SteamVR is kept active throughout.)
3. Proton log has a huge stack trace ending with a stack overflow. It's mostly in system libraries, the only VR-related thing I see is libopenxr_loader.so.1 + 0x2a91d.
4. After the stack overflows, the Proton log consists of warn: VR state wait timeout (retrying) repeated each second until the game is killed.

Also note that native VR applications (tested Half-Life: Alyx [546560] and native vrcmd), non-VR Proton games (tested Terraria [105600] and Submachine: Legacy [1564790]), and Proton games which can launch with or without VR (tested Beat Saber, VRChat and SCP: Secret Laboratory, all with SteamVR closed) all work as expected.

Reproduction

1. Install SteamVR beta, Proton experimental, and any of the games above.
2. (for vrcmd) Add vrcmd.exe from a Windows SteamVR installation as a non-Steam game and set the launch arguments to --spamhaptics.
3. Set the chosen application to use Proton Experimental and configure user_settings.py to log with +vrclient.
4. Open SteamVR and confirm that the VR environment is rendered and working correctly.
5. Launch the game (from desktop Steam client or from VR).

Logs

Collected with "WINEDEBUG": "+timestamp,+pid,+tid,+seh,+unwind,+threadname,+debugstr,+loaddll,+mscoree,+vrclient" in user_settings.py (added +vrclient, otherwise unmodified from user_settings.sample.py).

Warning, each Proton log extracts to ~600MB. Beat Saber: steam-620980.log.gz SCP: Secret Laboratory: steam-700330.log.gz VRChat: steam-438100.log.gz vrcmd.exe: steam-11877923831706288128.log.gz

[Bitwolfies]

Agreed, proton now simply fails to launch the vast majority of VR games properly, I've gotten Moss and Half Life 2 VR to work, but Half Life 2 VR's controls are completely broken.

[peterfab9845]

In order to eliminate as many variables as possible, I wrote a couple of test programs using the OpenVR and OpenXR APIs which simply initialize the API, wait 10 seconds, and then shut down: openvr_test.cpp.txt openxr_test.cpp.txt

1. If I build and run these natively on Linux, they work fine as expected.
2. If I build and run these natively on Windows, they work fine as expected.
3. If I take those Windows executables and try to run them via Proton on Linux: a. The OpenVR test works as expected. b. The OpenXR test demonstrates the same behavior as described in the initial report (stack overflow loading openxr_loader.so, followed by repeated "VR state wait timeout"). The test never prints any of the markers later than "preinit".

Edit: adding logs OpenVR log (400M): steam-11463073133859176448.log.gz OpenXR log (600M): steam-10877733739262115840.log.gz

[peterfab9845]

Tested with several Proton versions. All used Steam beta client 1699401499 and SteamVR beta 2.1.1.

	Beat Saber	The Lab	OpenVR test	OpenXR test	SCP:SL	VRChat	vrcmd.exe
AppID	620980	450390	11463073133859176448	10877733739262115840	700330	438100	11877923831706288128
Proton experimental-8.0-20231107	Fails as initial report	Fails as initial report	Works	Fails as initial report	Fails as initial report	Fails as initial report	Fails as initial report
Proton 8.0-4c	Fails as initial report	Fails as initial report	Works	Fails as initial report	Fails as initial report	Fails as initial report	Fails as initial report
Proton 8.0-4c try 2	Fails as initial report	Fails as initial report	Works	Fails as initial report	Fails as initial report	Fails as initial report	Fails as initial report
local 8.0-1c	Hangs at the stack overflow	Fails as initial report	Works	Fails as initial report	Fails as initial report	Hangs at the stack overflow	Fails as initial report
Proton 7.0-6e	Works	Works	Works	Works	Unity crash	Unity crash	"Don’t recognize interface name"
Proton 7.0-6e try 2	Works	Works	Works	Works	Crash if HMD moved	Works	"Don’t recognize interface name"
local 7.0-6e	Works	Works	Works	Works	Crash if HMD moved	Works	"Don’t recognize interface name"

Both games which had Unity crashes in Proton 7.0-6 launch fine in non-VR mode with Proton 8.0-4 and Experimental, and still fail in non-VR mode with 7.0-6, so I believe the 7.0-6 failures are unrelated and/or fixed in later Proton versions. Beat Saber is on the "beta" legacy1.29.1_unity_ver2019.4.28f1 branch, so maybe this Unity version isn't affected.

Edit 2023-11-09: Added second attempts for 8.0-4c and 7.0-6e, a local build of 7.0-6e, and a local build of 8.0-1c.

[Seercat3160]

I'm having the same issue, where OpenXR-based VR works in Proton 7.0-6 but not in Proton 8.0-1 or later.

I've only tested with VRChat (issue started when they switched to Unity 2022, I believe this is as unity2022 started doing some OpenXR stuff rather than OpenVR only) and No Man's Sky, but they both have the same characteristic printing to the Proton log: VR state wait timeout (retrying).

I've attempted to bisect Proton between those tags but haven't been able to do so due to build issues. The diff (incl submodules) is 3176468 lines so it's impractical for me to dig further.

System Information

• GPU: Nvidia RTX 3080 Ti
• Video driver version: nvidia-open 545.29.06
• Kernel version: 6.6.8.zen1
• Link to full system information report as Gist: https://gist.github.com/Seercat3160/0c3c76e5b865b40afc00d94bb01e8a00
  • In there you'll see reference to the fact that I'm launching Steam via prime-run: this is as my standard setup is Hyprland running on my iGPU and games run on my dGPU. However, I assure you that the issue in question still appears when I use the nvidia card as the primary GPU and run everything without PRIME.

[terminal-case]

LogOutput.log I think I'm having the same issue, in particular when trying to run Lethal Company with the VR mod. Attached is my most recent logs from an attempt just moments ago. Is there any version of Unity's OpenXR plugin that works with the latest SteamVR / beta, and with Proton 8? I might fork the VR mod myself if changing the version of OpenXR plugin used by the mod can get it to work.

[peterfab9845]

Tried the test programs again with new versions of everything. Client: Steam Beta Client 1706155871 SteamVR: beta Version 2.3.4 (1706131326) Graphics: nvidia 545.29.06 Kernel: 6.6.9-arch1-1

Results are the same as all other Proton 8 experiments with the following Proton installations.

Proton 8.0-5c

Proton: 1705700124 proton-8.0-5c
SteamGameId: 10877733739262115840
Command: ['/mnt/win/Users/Peter/Downloads/vrtest/oxr_test/main.exe']
Options: {'forcelgadd'}
depot: 0.20231211.70175
pressure-vessel: 0.20231208.0 scout
scripts: 0.20231208.0
sniper: 0.20231211.70175 sniper 0.20231211.70175
Kernel: Linux 6.6.9-arch1-1 #1 SMP PREEMPT_DYNAMIC Tue, 02 Jan 2024 02:28:28 +0000 x86_64
Language: LC_ALL None, LC_MESSAGES None, LC_CTYPE None
Effective WINEDEBUG: +timestamp,+pid,+tid,+seh,+unwind,+threadname,+debugstr,+loaddll,+mscoree

Proton Experimental 8.0-20240111c

Proton: 1705004294 experimental-8.0-20240111c
SteamGameId: 10877733739262115840
Command: ['/mnt/win/Users/Peter/Downloads/vrtest/oxr_test/main.exe']
Options: {'forcelgadd'}
depot: 0.20231211.70175
pressure-vessel: 0.20231208.0 scout
scripts: 0.20231208.0
sniper: 0.20231211.70175 sniper 0.20231211.70175
Kernel: Linux 6.6.9-arch1-1 #1 SMP PREEMPT_DYNAMIC Tue, 02 Jan 2024 02:28:28 +0000 x86_64
Language: LC_ALL None, LC_MESSAGES None, LC_CTYPE None
User settings WINEDEBUG: +timestamp,+pid,+tid,+seh,+unwind,+threadname,+debugstr,+loaddll,+mscoree,+vrclient
Effective WINEDEBUG: +timestamp,+pid,+tid,+seh,+unwind,+threadname,+debugstr,+loaddll,+mscoree,+vrclient

[peterfab9845]

Replying to https://github.com/ValveSoftware/Proton/issues/7228#issuecomment-1911214557

I suspect this might be a different issue. In the log you provided, the game seems to at least get past the OpenXR initialization; on lines 26+, I see Initialized OpenXR Runtime, and then it successfully requests the OpenXR version: OpenXR runtime being used: SteamVR/OpenXR (2.3.4). Just based on the errors, it seems like something related to input rather than the initial setup.

Looking at the mod's code, I think this is the line equivalent to my test program: https://github.com/DaXcess/LCVR/blob/f16127bb49aa947556c362d19379250c08ca122a/LCVR/Plugin.cs#L313

@terminal-case, could you set the game's launch options to PROTON_LOG=1 %command% and check the resulting steam-<numbers>.log in your home dir? If you see "VR state wait timeout" after a while, then it's probably the same, but if not, you might have better luck opening a new issue.

[Hisanatos]

I suspect this may be some bug in Nvidia driver so I opened an issue on their forums. https://forums.developer.nvidia.com/t/entire-graphics-stack-crashes-when-opening-a-steamvr-application-through-proton-compatiblity-layer-that-targets-openxr-rtx-3060-550-76-cuda-12-4/291155