Proton cant create new prefixes on Proton 8 and Experimental

[jkrhu]

Hi!

For some reason Proton is unable to create new prefixes for installed games when using Proton 8 or Experimental. I've tested a few games and every one of them is stuck during the prefix creation. I can wait minutes with no activity. This is with Experimental

GameAction [AppID 311340, ActionID 1] : LaunchApp changed task to ProcessingInstallScript with ""
Proton: Upgrading prefix from None to 8.0-102 (/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/)
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely

This is with Proton 8.0-2

GameAction [AppID 311340, ActionID 1] : LaunchApp changed task to ProcessingInstallScript with ""
Proton: Upgrading prefix from None to 8.0-101 (/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/)
Proton: Error while copying to "/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/pfx/drive_c/windows/system32/d3d12core.dll": No such file or directory
Proton: Error while copying to "/mnt/NVME2/Gry/SteamLibrary/steamapps/compatdata/311340/pfx/drive_c/windows/syswow64/d3d12core.dll": No such file or directory
fsync: up and running.
wine: RLIMIT_NICE is <= 20, unable to use setpriority safely

The games will successfully create a Prefix when using the Proton 7.0-6. They will also successfully upgrade, if the prefix was created in an earlier version. Another major annoyance is that even after cancelling the operation and closing Steam, the process will relaunch itself in the background with a winedevice, install script evaluator and a few other processes. No matter how many times I kill them, they return and require a reboot. Without it I can't launch any other Steam game.

My specs:

Fedora Linux 38 (6.3.8-200.fc38.x86_64) Intel i5-10400 16GB RAM Nvidia RTX 3060 (535.54.03)

Steam PC System Information

[pchome]

Still guessing, but as temporary solution people could try to rename the installscript.vdf file inside the game's directory (the one contains DXSETUP). In most cases those installations are not required on linux. Or just copy d3dx*.dll files from the other compatdata prefix directory (pfx/drive_c/windows/{system32,syswow64}/).

Or another variant: copy missed dlls into e.g. /opt/dlls/{x32,x64}/ and use WINEPATH="Z:\\opt\\dlls\\x32;Z:\\opt\\dlls\\x64". But this will work only for missing files, or otherwise wine's d3dx*.dll will be used.

I guess from proton's side there could be an option to ignore installation process, or maybe they could bundle d3dx* and use the WINEPATH trick.

[reedlove]

Still no update? Not being able to reliably create a prefix using the latest version of Proton seems like a pretty big deal. I know that everything wrong under the sun with Linux graphics is NVIDIA's fault and all (according to everyone), but somehow many games and other pieces of software still seem to work just fine using their proprietary drivers.

[ralgar]

@reedlove Valve neither has control over DXSETUP nor the Nvidia driver, so it would seem likely there isn't a lot they can do on this one.

I doubt Microsoft will lift a damn finger unless this affects Windows as well, so it's basically up to Nvidia to trim their driver back down, and with their pace of development on Linux who knows how long that will take.

This is the unfortunate downside of translation layers - we are still beholden to Microsoft and their ecosystem in some ways.

---

I do wonder if a LAA patch, like we used to apply to 32 bit Skyrim, might keep it from crashing. Has anybody looked into such a thing?

I'm not especially hopeful, but I might try it this weekend if I get a chance.

[kisak-valve]

PROTON_FORCE_LARGE_ADDRESS_AWARE has been enabled by default for a fair while now, and should apply to everything 32 bit.

[ralgar]

@kisak-valve Had a feeling y'all would've done something to this effect. Thanks for confirming :)

[reedlove]

Replying to https://github.com/ValveSoftware/Proton/issues/6859#issuecomment-1710939726

I'm aware and am being a bit sarcastic. ;] Also, I thought that large address awareness pertained to system memory addressing limitations and not necessarily gpu memory addressing as I thought it would be handled via the driver and graphics API.

[ralgar]

@reedlove Gotcha. Sometimes it's difficult to tell in text :)

Unless I'm grossly misunderstanding the issue, as laid out above by Ivy, this is related to DXSETUP exhausting its addressable system memory. You are otherwise correct in your assertion though.

[liberodark]

Hi, hope we find solution about this.

Best Regards

[ElmerFuddDK]

I too am experiencing issues with proton not creating prefixes. I'm on MX-Linux (debian 12 based) on an AMD based SoC, so fully mesa.

I tried exporting various combos of the following before starting: export VK_ICD_FILENAMES= export PROTON_FORCE_LARGE_ADDRESS_AWARE=1 export PROTON_LARGE_ADDRESS_AWARE=1 export WINE_LARGE_ADDRESS_AWARE=1

Each time the prefix creation process hang with the process: C:\windows\syswow64\rundll32.exe setupapi,InstallHinfSection Wow64Install 128 \?\Z:...\usr\share\wine\wine.inf Killing this would exit the startup with a failure: wine: could not load kernel32.dll, status c0000135

I tried with proton 8.0-c and experimental-8.0-20230907

Wine 8.12-64bit-staging works fine. Proton 7 works fine.

[ivyl]

Current Nvidia beta driver 535.43.09 seems to help with the memory usage significantly. Me and other people who had local repro are no longer able to hit the problem even when having both Mesa and Nvidia blobs installed and loaded into the address space.

We are still looking for potential address space savings inside Wine / Proton but it's unlikely that we'll find anything significant.

FWIW there's upstream work to have working wow64 in Wine - the Windows part will run in 32bits but the Linux side will be full on 64bits. There's still ways to go to have it complete and working well, we are talking multiple Proton rebases away.

@ElmerFuddDK Please verify your proton installation. You issue seems unrelated to the OOM problem. wine: could not load kernel32.dll, status c0000135 means the DLL was not found. That's usually a symptom of busted Proton installation.

[jkrhu]

For me, I had no problems with it since 535.98. I'm on default Fedora 38, 535.104.05 drivers and it's solid on my 3060. Grub settings disable nouveau by default and I also enable the nvidia-drm.modeset. I still have mesa packages installed. Not sure why it only works for some users now.

[ElmerFuddDK]

@ivyl that error is shown when I kill the hanging process, and then proton exits. I just did a re-test and proton can create prefixes from directly from steam now. I'm running proton "command line", either from the installed proton or via proton-GE in lutris. You may be right that it is not related to this issue. It's might be an odd coincidence it hangs in syswow64 initialization.

[jinzhongjia]

now, I also meet this problem on archlinux, and my laptop has an Intel CPU and NVIDIA graphics card on Deep Rock Galactic, I must first create the prefix through proton7 and then start it through proton8

[pellcorp]

Can confirm rolling back to NVIDIA 525 drivers on Ubuntu 22.04 fixes the dxsetup issue.

Previously on 535 I would get the error for most games I would try to install, seems like perhaps there might be timing issues here too, but now with 525 it's consistently installing games again, which is a relief as I reinstalled my sons Ubuntu and steam and wanted to re-download all his big games so it would be ready for him when he got back, having to explain that you need to wait say 10 minutes and then kill the dxsetup process from monitoring would have got old real quick

Hopefully a better solution with 535 drivers will be provided

[Jan200101]

Also managed to recreate this with Titanfall 2 and the EA Desktop Installer

[psycowithespn]

I believe I ran into this on NixOS Unstable, Nvidia 980ti. Game was Killing Floor 2. I was using nixpkgs#steam.

However, I believe I did NOT run into this when I tried Flatpak steam with the same game, same Proton version, and same version of nixpkgs. Is there anything specific I can capture to help with this?

[jinzhongjia]

Ultra Street Fighter IV can also meet this problem

[chiat88]

Glad this thread is active. Valve support mentioned they only support Ubuntu among all linux distros.

Fedora 37 + nvidia user here. Proton used: 8.0-4, and Proton Experimental nvidia driver: 535

Just to add log on Octopath Traveler 2 stuck at DXSETUP.exe. After manually killing it, the game will launch. I am expecting launcher should be able to bypass DXSETUP.

It frustrated me as it won't produce $PROTON_LOG at all (cause it is not even starting). *Note: Workaround on clearing download cache worked 1 time or 2, problem will repeat again.

[ralgar]

Just chiming in again to confirm that this appears to finally be resolved with driver 545.

[4nexus5]

Any new updates or solutions? I am on AMD GPU and the issue still persists. None of the solutions I found online worked for me (disabling shader pre-caching, reinstalling, downloading additional drivers etc.). I am on Arch Linux

[jinzhongjia]

This problem seems not happen on myself now. Maybe you can try proton9-ge? or proton expermental ?

[4nexus5]

This problem seems not happen on myself now. Maybe you can try proton9-ge? or proton expermental ?

Nope, switched between multiple Proton versions, the issue still persists

[jkrhu]

This problem seems not happen on myself now. Maybe you can try proton9-ge? or proton expermental ?

Nope, switched between multiple Proton versions, the issue still persists

This is a different issue. The topic is about the Nvidia propietary driver and DirectX redistributable causing exhaustion of 32bit memory address space. Which prevented the prefix from being created.

I would suggest to open a new issue if you have similar issues on RADV. The Nvidia issue has been solved since then.

@kisak-valve @ivyl Should we close this Nvidia issue?