Proton cant create new prefixes on Proton 8 and Experimental

[jkrhu]

Hi!

For some reason Proton is unable to create new prefixes for installed games when using Proton 8 or Experimental. I've tested a few games and every one of them is stuck during the prefix creation. I can wait minutes with no activity. This is with Experimental

GameAction [AppID 311340, ActionID 1] : LaunchApp changed task to ProcessingInstallScript with ""
Proton: Upgrading prefix from None to 8.0-102 (/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/)
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely

This is with Proton 8.0-2

GameAction [AppID 311340, ActionID 1] : LaunchApp changed task to ProcessingInstallScript with ""
Proton: Upgrading prefix from None to 8.0-101 (/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/)
Proton: Error while copying to "/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/pfx/drive_c/windows/system32/d3d12core.dll": No such file or directory
Proton: Error while copying to "/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/pfx/drive_c/windows/syswow64/d3d12core.dll": No such file or directory
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely

The games will successfully create a Prefix when using the Proton 7.0-6. They will also successfully upgrade, if the prefix was created in an earlier version. Another major annoyance is that even after cancelling the operation and closing Steam, the process will relaunch itself in the background with a winedevice, install script evaluator and a few other processes. No matter how many times I kill them, they return and require a reboot. Without it I can't launch any other Steam game.

My specs:

Fedora Linux 38 (6.3.8-200.fc38.x86_64) Intel i5-10400 16GB RAM Nvidia RTX 3060 (535.54.03)

Steam PC System Information

[Plarpoon]

I can confirm having the same issue, but this is valid only for certain games.

For example:

• Titanfall
• Deep Rock Galactic both suffer from this issue, and as you have mentioned it's only happening if you run them the first time directly with Proton 8, that is not happening if you had previously successfully executed them with Proton 7 or prior.

Same issue is not present instead for such games like:

• Orcs Must Die 3 Which works without any issues at first execution directly with Proton 8.

Important to say that I had none of these issues when running these games directly under Proton 8 on the Steam Deck,these are issues I have always had exclusively with regular Linux on my PC.

This was tested on both Fedora 38 and NixOS.

This issue I had created I just discovered actually being related to this specific problem and not to only Titanfall.

https://github.com/ValveSoftware/Proton/issues/4410#issuecomment-1586400051

[jkrhu]

I would like to also point out that this problem is occurring only on the Steam client. I can still create new prefixes with new Wine and Proton versions from Bottles or Heroic Games Launcher. I would assume it's connected to some of the changes in the new client recently. I've been testing the beta some time ago and it was fine then, but not the more recent updates.

[DownloadFreeloader]

I believe I may be having a similar issue. Recently I have been trying Next Fest demos and they have all been following a pattern where they will not launch with Proton 8 or Experimental (will say launching, running, then instantly close), will fail to actually run with Proton 7 (will get stuck at either launching or running), but will then work normally after switching back to 8. Using the Thronefall Demo as an example, this is what was output by PROTON_LOG=1 on the initial launch with Proton 8: https://pastebin.com/qxhibq0z When switching to Proton 7, this was the log output, with the last line I believe being caused by me clicking "Stop" in the Steam client: https://pastebin.com/nFqw9gCE

[Plarpoon]

Yes, that is 100% compatible with the same issues I was having too. Which sucks because I am not sure it will be an easy fix.

[jkrhu]

Today after the SELinux policy update on Fedora 38, Proton 8 and Experimental started working for me again.

[TheEpicNoobZilla]

Fedora 38 and still not working. Had to delete prefix made by Proton 8 (both Valve's and GE)/Experimental then launch the game with proton 7 or older to create prefix that actually launches the game. Proton log gives no information that would help since it fails to launch the game and despite steam detecting this state as in game

[ralgar]

I'm also suffering a similar issue on Arch Linux. The only difference seems to be that Proton 8.0 refuses to work at all - upgrading the prefix from Proton 7.0 doesn't work for me. It just hangs on "Launching" indefinitely. The various processes also won't terminate when stopping the game, shutting down Steam, or logging out - forcing me to reboot the system.

I had exactly the same log output as @jkrhu the other day, however with the new client update today I get more verbose output:

steamlog.txt

I also managed to get a Proton log today, indicating that it is at least trying to start the game process now:

steam-782330.log

@kisak-valve Can we get some eyes on this please? It's a pretty major issue. Like @jkrhu, I suspect this is caused by the new Steam client itself, as I had no issues with Proton 8.0-2 prior to the recent Steam client updates, nor have I ever had issues in the last 4 years I've been running this setup.

[Plarpoon]

I agree, this is definitely something major to look into. It's distro independent and seems to ubiquitously exist across all minor versions of Proton 8. As of now I had the issue in NixOS, Fedora 38, Fedora 39 and PopOS!

[jkrhu]

For me the problem stopped occuring after recent updates. I have a hunch it might be driver dependent too. I was initially on 535, but had problems with screen tearing. Eventually rolled back to 530, I kinda left it since there was no development on the issue. There were a bunch of updates like SELinux and when I tried a fresh prefix, it started working again. What drivers are you running? Have you tried with 530? I think the 535 has massive issues. The shader cache is broken, it has screen tearing on Wayland etc.

[alasky17]

Is anyone who is experiencing this issue not on NVIDIA with 535 drivers? Someone who was experiencing something that sounded exactly like the issue described was able to narrow it down to the 535 drivers, and rolling back to 530 fixed the issue. @jkrhu it sounds like at least in your case, this was the problem?

[Plarpoon]

Several of us have that issue on full AMD setups, so I am doubtful it is just a driver issue. Maybe some internal dependency.

[alasky17]

@Plarpoon Thank you. I saw your post a while ago and tested Deep Rock Galactic locally, and could not repro following your steps :( (also with an all AMD setup with Arch). To start with, could you copy your system info from your original comment on the Titanfall bug so that it is all in one place?

Then, there are a couple of things for you to try: 1) Please update to Mesa 23.1.3 if you are still on 23.1.1. 2) Please disable gamemode and manguhud entirely. 3) If you are still getting failures with a fresh prefix for any/all of the games mentioned after completing these steps, could you please get a log using "PROTON_LOG=+vulkan %command%" for each failing game?

[Plarpoon]

1) I am running always on the latest package updates, I am sort of an update freak :) 2) I did run without any parameter and it didn't change much unfortunately 3) steam-548430.zip Deep Rock Galactic

I unfortunately don't have Titanfall 1 installed anymore as it was quite quick to complete and go to Titanfall 2. But I will post more updates whenever something shows up!

Thanks for the interest @alasky17, anything else you need just let me know and I will provide it. Even if it's boring to do, I just want to solve this issue!

P.S. For the DRG log I uploaded I had re-enabled gamemode, I did test it without before but it still didn't work!

[alasky17]

@Plarpoon This is indeed boring :P , but could you please get a couple more logs? 1) It would be useful to get a log with gamemode disabled, just on the off chance that there are two failures going on, and disabling gamemode changes the failure. 2) Could you get a log of the game launching correctly on Proton 7.0-6 for comparrison?

[Plarpoon]

@alasky17

DRG - Proton 7.0-6 - No extra parameters the game without gamemode is a lag fest when all in Ultra, absolutely terrible performance, but playable.

DRG - Proton 8.0-2 - No extra parameters unlike the version with 7.0-6 that has virtually no loading screen as they are extremely fast, the one with 8.0-2 actually takes approximately one entire minute after which it crashes to desktop (never managing to give me an actual playable character). After that for another minute Steam thinks I am actually still in-game despite the game being already crashed, it's only after this extra time that the option to "Play" the game comes available again! (so the process remains open despite the crash for some time)

These are the launch settings I used

[alasky17]

@Plarpoon I think I'm a bit confused - I read your comment "both suffer from this issue, and as you have mentioned it's only happening if you run them the first time directly with Proton 8, that is not happening if you had previously successfully executed them with Proton 7 or prior." and took that to mean that the game was failing to launch when the prefix was created on Proton 8.0-x but the game works when the prefix is created on Proton 7.0-x and then upgraded. Does creating the prefix on Proton 7.0-6 and then upgrading fix this issue with DRG? Also, it sounds like the failure happens after a window is created?

[Plarpoon]

@alasky17 That's the odd thing, Titanfall 1 that I don't have installed anymore used to behave that way. If you created the symlinks with proton 7 then run it with 8 it would work, DRG on the other hand it works only with Proton 7.

And yes, I confirm that, the failure in Proton 8 happens well after the window is created. There is first a splashscreen and then a loading screen, in that loading screen it dies. On Titanfall it wouldn't even boot instead.

[runar-work]

I was also unable to reproduce any Deep Rock Galactic launch issues, but I can reproduce the Titanfall problem, so it seems like those are separate issues. Titanfall Proton log after a successful launch is absolutely massive, though.

• Kernel: 6.4.3-arch1-2
• GPU: RX 7600
• Driver: 23.1.3

[alasky17]

@Plarpoon It sounds like we should move the Deep Rock Galactic issue back to https://github.com/ValveSoftware/Proton/issues/284 because it is a game-specific issue, not a general problem, and it is not related to creating the prefix on Proton 7 vs Proton 8. Could you please write a comment there with your system specs, the current behavior on Proton 7 and Proton 8, and the logs you collected without gamemode or manghud to make this easier to follow?

[runar-work]

I decided to test Monster Hunter Rise installation again, just to be absolutely certain it wasn't the same problem, and it turns out I get the same crash in the prefix installation even after I've switched to an AMD GPU. I just need to have the Nvidia driver installed, and the prefix installation will still fail with the same crash in libnvidia-glcore.so.535.54.03. After removing the driver and rebooting, it installed normally again.

For a brief while I could no longer reproduce the Titanfall issue. I deleted the prefix and it still launched normally with Proton 8.0. Now it's back to failing again. I collected various logs, mostly failures to launch and one from a successful 8.0 launch. The Proton log easily reaches 500 MB before the game even launces, so I just shut down the game via the Steam UI once it launched to keep it a bit shorter.

Titanfall_launch_issues.zip

[jkrhu]

@alasky17 Yes, I would say that the 535 series driver was the culprit in my case. Works fine on 530.

[alasky17]

@runar-work wrt the nvida-535 crash still happening on AMD as long as the library is installed; strange! Thank you for the update, and it sounds like we need to continue to investigate this further.

@Plarpoon Thank you for the consolidated report on the DRG issue; greatly appreciated!

@jkrhu Thank you for the confirmation :)

---

@runar-work @Plarpoon wrt Titanfall - we are also looking into this now and can repro but no leads on a cause so far. This seems like it is also game-specific and not related to the nvidia-535 library issue, so I think it will make more sense for future discussion to happen on the game thread following up on @Plarpoon's original comment here: https://github.com/ValveSoftware/Proton/issues/4410#issuecomment-1586400051

---

As far as I can tell, the original issue in this thread is the nvidia 535 library issue causing prefix creation failure, which is apparently not specific to NVIDIA GPUs but rather the library itself. Any other issues that happen without the nvidia 535 library installed on a system seem to be game specific and are being shuttled to the game threads. Since the NVIDIA driver issue has not been solved, I think it makes sense to keep this thread open to track that specifically.

Thanks to everyone for the responsiveness to help to detangle this issue. It is greatly appreciated :)

[Plarpoon]

Thanks to you man!

[jkrhu]

Problem is still occuring on the 535.86.05 driver

[ivyl]

Nvidia has been notified of this on 14th of July.

It looks like some regression in their 32bit driver. When DXSETUP.EXE is being run it crashes in the Nvidia driver (/usr/lib/pressure-vessel/overrides/lib/i386-linux-gnu/libnvidia-glcore.so.535.54.03 at offset 0x1739e1d) on the Unix side. It started happening with this driver version specifically and it still works okay with an older one.

It crashes when we are calling vkCreateInstance() in winex11.drv. Things that get passed via all the Wine layers to the actual driver look correct. Vulkan Validation Layers also did not complain.

I wasn't able to repro with 535.86.05 though. Does the issue still persist for you folks?

[reedlove]

@ivyl Are you able to explain why it works in Proton 7 and not 8? Using the 535.54.03 drivers, that is.

[runar-work]

Yeah, I still have the same problem. Here's what I got with Vampyr and Proton 8.0 with 535.86.05:

24371.668:0324:0328:warn:seh:handle_syscall_fault backtrace: --- Exception 0xc0000005 at 0x6f26442d: /usr/lib/pressure-vessel/overrides/lib/i386-linux-gnu/libnvidia-glcore.so.535.86.05 + 0x173b42d.

[ivyl]

@ivyl Are you able to explain why it works in Proton 7 and not 8? Using the 535.54.03 drivers, that is.

No idea. Not much has changed on the Wine side and I don't know what the driver is doing, it's proprietary.

Since the older drivers were working just fine with 8.0 for months, other vendors do not have the same problem and validation layers do not complain it seems more productive to focus on why the driver has regressed.

[jkrhu]

Does the issue still persist for you folks?

Yes, it does. Still requires the Proton 7 workaround during prefix creation. Upgrading works fine. The 535 series is very unfortunate. Major regressions in different spots. Hope it gets sorted out soon.

[ralgar]

@ivyl Can confirm the issues persist for me, as of last night. Thanks for confirming the driver as the cause.

[adamnv]

Can someone facing this issue please post the output of running 'getenforce' or 'grep SELINUX= /etc/selinux/*|grep -v #'? Thanks!

[reedlove]

Actually, it returns nothing.

The only uncommented lines are:

module-store = direct expand-check=0

On Linux Mint 21.2

[adamnv]

Thanks, @reedlove - I assume you don't have getenforce? To the best of your knowledge you're not running with selinux enabled? :) That'd rule that out, then...

[reedlove]

Thanks, @reedlove - I assume you don't have getenforce? To the best of your knowledge you're not running with selinux enabled? :) That'd rule that out, then...

Correct, selinux is not enabled. XD hah

[Poslovitch]

Still requires the Proton 7 workaround during prefix creation

The workaround doesn't work on my end : "compatibility tool failed" with Proton 7-6.

[jkrhu]

Still requires the Proton 7 workaround during prefix creation

The workaround doesn't work on my end : "compatibility tool failed" with Proton 7-6.

Did you remove the compatdata game subfolder before forcing Proton 7.0-6?

[Poslovitch]

Yup. But Proton doesn't even try to create a prefix: it seems to fail before that. At least, I don't see Proton talking about prefixes in the logs.

And there's the log when trying to load the game with Proton 8 (after trying to launch it with 7):

Proton: Upgrading prefix from None to GE-Proton8-9 (/software/steam/steamapps/compatdata/2239150/)
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely

Okay, err... Did Ctrl+C and here's what it printed out:

^Csrc/clientdll/installscript_posix.cpp (531) : Assertion Failed: Standalone evaluator returned error code for app 2239150
assert_20230803211806_37.dmp[41149]: Uploading dump (out-of-process)
/tmp/dumps/assert_20230803211806_37.dmp
/bin/sh\0-c\0PROTON_LOG=1 /home/poslovitch/.steam/debian-installation/ubuntu12_32/reaper SteamLaunch AppId=2239150 -- /home/poslovitch/.steam/debian-installation/ubuntu12_32/steam-launch-wrapper -- '/home/poslovitch/.steam/debian-installation/steamapps/common/SteamLinuxRuntime_sniper'/_v2-entry-point --verb=waitforexitandrun -- '/home/poslovitch/.steam/debian-installation/compatibilitytools.d/GE-Proton8-9'/proton waitforexitandrun  '/software/steam/steamapps/common/Thronefall/Thronefall.exe'\0
chdir "/software/steam/steamapps/common/Thronefall"
ERROR: ld.so: object '/home/poslovitch/.steam/debian-installation/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/poslovitch/.steam/debian-installation/ubuntu12_64/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
ERROR: ld.so: object '/home/poslovitch/.steam/debian-installation/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/poslovitch/.steam/debian-installation/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/poslovitch/.steam/debian-installation/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
WARNING: discarding _NET_WM_PID 2 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
WARNING: discarding _NET_WM_PID 2 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
WARNING: discarding _NET_WM_PID 2 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
WARNING: discarding _NET_WM_PID 2 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
assert_20230803211806_37.dmp[41149]: Finished uploading minidump (out-of-process): success = yes
assert_20230803211806_37.dmp[41149]: response: CrashID=bp-006665b8-c2f5-4fe5-aff0-edb6d2230803
assert_20230803211806_37.dmp[41149]: file ''/tmp/dumps/assert_20230803211806_37.dmp'', upload yes: ''CrashID=bp-006665b8-c2f5-4fe5-aff0-edb6d2230803''
ProtonFixes[41274] INFO: Running protonfixes
ProtonFixes[41274] INFO: Running checks
ProtonFixes[41274] INFO: All checks successful
ProtonFixes[41274] INFO: Using global defaults for "Thronefall" (2239150)
ProtonFixes[41274] INFO: No protonfix found for "Thronefall" (2239150)

It's still not running, but Steam shows it as such (which is somehow better than it being stuck on "Launching...", I guess?).

[jkrhu]

The issue is fixed on Nvidia 535.98 driver. Games which wouldn't create a prefix under Proton 8 previously on the 535 series, do work correctly now on my system.

Proton: Upgrading prefix from None to 8.0-104 (/mnt/NVME/Games/Steam Library/steamapps/compatdata/311340/)
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely
/bin/sh\0-c\0PROTON_LOG=1 /home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/reaper SteamLaunch AppId=311340 -- /home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/steam-launch-wrapper -- '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/SteamLinuxRuntime_sniper'/_v2-entry-point --verb=waitforexitandrun -- '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Proton - Experimental'/proton waitforexitandrun  '/mnt/NVME/Games/Steam Library/steamapps/common/Metal Gear Solid Ground Zeroes/MgsGroundZeroes.exe'\0
chdir "/mnt/NVME/Games/Steam Library/steamapps/common/Metal Gear Solid Ground Zeroes"
ERROR: ld.so: object '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
ERROR: ld.so: object '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/jkrhu/.var/app/com.valvesoftware.Steam/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
WARNING: discarding _NET_WM_PID 5 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
WARNING: discarding _NET_WM_PID 5 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
WARNING: discarding _NET_WM_PID 5 as invalid for X11 window - use specialized XCB_X11_TO_PID function!
08/11 22:45:21 Init: Installing breakpad exception handler for appid(gameoverlayui)/version(20230728184426)/tid(1924)
                                     08/11 22:45:21 Init: Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)/tid(1924)
                                                               Game 311340 created interface STEAMAPPLIST_INTERFACE_VERSION001 / AppList
Game 311340 created interface STEAMAPPS_INTERFACE_VERSION006 / Apps
Game 311340 created interface STEAMAPPTICKET_INTERFACE_VERSION001 / 
Game 311340 created interface STEAMCONTROLLER_INTERFACE_VERSION / Controller
Game 311340 created interface STEAMHTMLSURFACE_INTERFACE_VERSION_002 / HTMLSurface
Game 311340 created interface STEAMHTTP_INTERFACE_VERSION002 / HTTP
Game 311340 created interface STEAMMUSICREMOTE_INTERFACE_VERSION001 / MusicRemote
Game 311340 created interface STEAMMUSIC_INTERFACE_VERSION001 / Music
Game 311340 created interface STEAMREMOTESTORAGE_INTERFACE_VERSION012 / RemoteStorage
Game 311340 created interface STEAMSCREENSHOTS_INTERFACE_VERSION002 / Screenshots
Game 311340 created interface STEAMUGC_INTERFACE_VERSION003 / UGC
Game 311340 created interface STEAMUNIFIEDMESSAGES_INTERFACE_VERSION001 / UnifiedMessages
Game 311340 created interface STEAMUSERSTATS_INTERFACE_VERSION011 / UserStats
Game 311340 created interface SteamFriends014 / Friends
Game 311340 created interface SteamMatchMaking009 / Matchmaking
Game 311340 created interface SteamMatchMakingServers002 / MatchmakingServers
Game 311340 created interface SteamNetworking005 / Networking
Game 311340 created interface SteamUser017 / User
Game 311340 created interface SteamUtils007 / Utils
Game 311340 method call count for IClientRemoteStorage::FileExists : 6
Game 311340 method call count for IClientRemoteStorage::GetFileTimestamp : 6
Game 311340 method call count for IClientRemoteStorage::GetFileSize : 2
Game 311340 method call count for IClientRemoteStorage::FileRead : 1
Game 311340 method call count for IClientRemoteStorage::FileWrite : 2
Game 311340 method call count for IClientUserStats::GetAchievementDisplayAttribute : 2
Game 311340 method call count for IClientUserStats::GetAchievement : 1
Game 311340 method call count for IClientUserStats::RequestCurrentStats : 1
Game 311340 method call count for IClientUtils::RecordSteamInterfaceCreation : 22
Game 311340 method call count for IClientUtils::GetAppID : 26
Game 311340 method call count for IClientUser::BIsSubscribedApp : 12
Game 311340 method call count for IClientUser::GetAppOwnershipTicketExtendedData : 1
Game 311340 method call count for IClientUser::GetUserDataFolder : 6
Game 311340 method call count for IClientUser::GetSteamID : 4
Game 311340 method call count for IClientUser::BLoggedOn : 2
Uploaded AppInterfaceStats to Steam 

[reedlove]

Just tried installing Tekken 7 using Proton 8 with nvidia-driver-535/jammy,now 535.98-0ubuntu0~gpu22.04.1 from the Ubuntu graphics team PPA and DXSETUP.exe is still freezing at first install.

[reedlove]

So explain this... If I go in to Downloads and clear the cache and then delete the compatdata folder and then run the game for the first time, DXSETUP.exe works just fine. Explain to me again how this is an nvidia problem and not a problem with Proton 8.0-3 or Steam?

[jkrhu]

In my own experience I've already installed a bunch of titles like MGS:GZ, Forspoken, WD2 and RE7. Every one of them successfully created a prefix after fresh install. I'm not sure why yours failed, perhaps some leftover files in compatdata from previous installations/prefixes. Could be also some incompatibility with trying to upgrade Proton 7 prefixes created using the previous 535 drivers.

[reedlove]

So now get this. AFTER I run Tekken 7, "Steamworks Common Redistributables" has an update. I update it. I go to run Overcooked! for the first time on this fresh install of Steam, using Proton 8.0-3. Guess what's hung again..... DXSETUP.exe

[reedlove]

In my own experience I've already installed a bunch of titles like MGS:GZ, Forspoken, WD2 and RE7. Every one of them successfully created a prefix after fresh install. I'm not sure why yours failed, perhaps some leftover files in compatdata from previous installations/prefixes. Could be also some incompatibility with trying to upgrade Proton 7 prefixes created using the previous 535 drivers.

This was a "first time" install of Tekken 7 on a fresh install of Steam on a fresh install of Linux Mint 21.2.

[reedlove]

Just cleared out the entire compatdata folder and reinstalled a number of games from scratch and everything appears to be working now. -_- Wacky.

[ivyl]

Good news - there's some progress. The issue seems to be due to address space exhaustion. DXSETUP is a 32 bit process and it can address 4G of memory. Nvidia's driver starting with 535 seem to be a bit more heavy than the previous series. While the bump is not colossal it's enough to make things crash. Nvidia is aware of this.

One thing that should help with the issue is uninstalling all the other Vulkan drivers that are not in use. So if you are on Nvidia uninstall all things Mesa, including extra Vulkan layers, and vice versa. Otherwise they are all loaded using the precious address space.

That said I'm looking into where the memory goes in places we have control over it and why DXSETUP is this pathological.

[reedlove]

Replying to https://github.com/ValveSoftware/Proton/issues/6859#issuecomment-1682357594

Yay! An explanation that appears to make sense! Good deal! Thanks for your hard work!

[pchome]

@ivyl What if you just disable Vulkan layers for the setup process?

Use VK_LOADER_DEBUG=layer (or =driver, =debug, =info, =all) to see what's going on. To disable known unwanted layers (steam layers (?), was enabled by steam client (?)) either set "DISABLE" environment variables for the process or blacklist them via a meta layer.

Meta layer example

Created by old `vkconfig` utility when on/off selected layers. `~/.local/share/vulkan/implicit_layer.d/VkLayer_override.json` ``` { "file_format_version": "1.1.2", "layer": { "api_version": "1.3.211", "blacklisted_layers": [ "VK_LAYER_VALVE_steam_fossilize_32", "VK_LAYER_VALVE_steam_fossilize_64", "VK_LAYER_NV_optimus", "VK_LAYER_VALVE_steam_overlay_32", "VK_LAYER_VALVE_steam_overlay_64", "VK_LAYER_MESA_device_select" ], "component_layers": [ ], "description": "LunarG Override Layer", "disable_environment": { "DISABLE_VK_LAYER_LUNARG_override": "1" }, "implementation_version": "1", "name": "VK_LAYER_LUNARG_override", "override_paths": [ ], "type": "GLOBAL" } } ``` This is **implicit** layer w/o `enable_environment` section so will be auto-loaded. **Add "ENABLE" variable to load it only when needed** (`enable_environment` section).

[EnLaLuna91]

Hello,

The same thing happens to me with Baldurs Gate 3. I've tried with Proton Experimental, Proton 8.0-3 and Proton GE.

Steam log

Harware info

I had been playing for days, until the Proton GE version was updated and since then I can't play, independently of the Proton version.

[reedlove]

Any news on this front?

[ArcadesSaboth]

I tried to log what layers I'm using as suggested by @pchome and I tried disabling them all, e.g.:

DISABLE_VK_LAYER_VALVE_steam_overlay_1=1 NODEVICE_SELECT=1 DISABLE_MANGOHUD=1 VK_LOADER_DEBUG=layer steam

So I ended up with no layers at all and I still can't create a prefix. I tested with Trover Saves the World, a UE4 game I happened to purchase yesterday and having this problem.

I would like to note that I am having a similar problem with The Elders Scrolls Online where the prefix is created but the installation of the supplied Microsoft VC++ 2010 hangs for no apparent reason on Proton 8 but works on Proton 7. I'm not sure if this problem is related to this problem. Additional note: for this game it hangs even if I try to install it with Protontricks.

I'm on Fedora 38, Nvidia drivers 535.98.