search

shkhln / linuxulator-steam-utils

Steam launcher for FreeBSD
MIT License
128 stars 12 forks source link

Proton (Steam Play) support #4

Closed darkfiberiru closed 3 years ago

darkfiberiru commented 4 years ago

Do you think this PR would be enough to allow steamplay

https://reviews.freebsd.org/D16830

shkhln commented 4 years ago

There is also steamclient.so, which is a closed source Linux binary used by Proton. I plan to extend nvshim to handle this use case.

shkhln commented 4 years ago

There is now a workaround (89fb6bb2a2a378084d6a82cf3a4df03eec9db078) for Linux Proton builds, since those are apparently partially capable of working under Linuxulator. Specifically, 64-bit Wine does work, and 32-bit Wine doesn't. (The latter is also responsible for running a couple of utility executables necessary for proper prefix setup, so YMMV.)

You'll also need the python3 and python3-libs packages from CentOS, see https://gist.github.com/shkhln/b5f9daf67d0a8a9b7d068d795eff68ad.

shuryanc commented 4 years ago

Hi Alex, do we need to run ./sglrun with steam in order to get steam proton to work?

Thanks.

shkhln commented 4 years ago

I don't think so, native Proton requires quite a bit more elaborate setup than that. On the other hand, Linux Proton doesn't need my shim.

shuryanc commented 4 years ago

Understood. Thanks.

shuryanc commented 4 years ago

Hi Alex,

Could you provide a tested proton game name? Just want to based on it to check if the setup is fine.

Thanks.

shkhln commented 4 years ago

want to based on it to check if the setup is fine.

Try this one. Should be as good as any.

shkhln commented 4 years ago

Try this one. Should be as good as any.

Look like this is a 32-bit game :raised_eyebrow: Fine, let's try the https://store.steampowered.com/app/848480/Creeper_World_4/ demo.

shuryanc commented 4 years ago

Got a memory dump, I am checking if its related to "wine: failed to initialize: libm.so.6: failed to map segment from shared object: Cannot allocate memory".

Memory Dump: https://gist.github.com/shuryanc/84f6c2874717f040177b751e72766407

shkhln commented 4 years ago

First, you are aware https://gist.github.com/ exists, right? Second, which steam-utils version is that?

shuryanc commented 4 years ago

Ah, I seldemly post to that link, I have edited the post to gist. I cloned the latest git, make and make install. Then ran /opt/steam-utils/bin/steam-install and /opt/steam-utils/bin/steam. So I think its the latest version.

BTW, I added /home/vagrant previously. If I remove it, the memory dump is: https://gist.github.com/shuryanc/cde8a073a04934dd6ce929f00e4fe05a

Need to test some rest. May check it again later.

shkhln commented 4 years ago

Verify that protonfix.so is actually accessible. Try 7c2f360dd02669726d8db9e9b3d3c07d99a22553.

shuryanc commented 4 years ago

Thanks Alex. This fixes it!

shkhln commented 3 years ago

There is also steamclient.so, which is a closed source Linux binary used by Proton. I plan to extend nvshim to handle this use case.

And this is mostly done, with the notable exception of the futex/robust_list stuff.

shkhln commented 3 years ago

There is now native Proton support. This requires 3 separate things:

  1. The emulators/wine-proton port/package, both 64- and 32-bit parts. For the latter you can use the provided helper script: lsu-pkg32 install wine-proton. The package will be downloaded and extracted into the ~/.i386-wine-pkg directory. [Packages aren't available yet, will be in a day or so.] Alternatively, you can build the port in a i386 chroot and install it there. In that case set the LSU_i386_PKG_ROOT environment variable to the chroot location. Something like echo "setenv LSU_i386_PKG_ROOT /compat/i386" >> ~/.cshrc && source ~/.cshrc.

  2. A certain shim library, https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250591.

  3. Proton 5.13 distribution, from which we need to copy some files. Everything other than Wine, actually. Download it in Steam, run lsu-register-proton, restart Steam.

shkhln commented 3 years ago

Obviously, this a very hit-and-miss solution with regard to the game compatibility, but we have to start somewhere.

darkfiberiru commented 3 years ago

So for the 32bit is only in the i386 tree. Could a slave port be done to build both in amd64 tree? I have my only person full package builds and only build amd64 right now.

Hmm I will play with this.

darkfiberiru commented 3 years ago

Feel free to close this item. Awesome work.

shkhln commented 3 years ago

So for the 32bit is only in the i386 tree. Could a slave port be done to build both in amd64 tree?

Not until portmgr@ allows it.

ericbsd commented 3 years ago

I did commit your new port emulators/libc6-shim to the FreeBSD ports tree.

ericbsd commented 3 years ago

I get this when I am starting a game, am I missing something?

Traceback (most recent call last):
    1: from /usr/local/steam-utils/lxbin/../bin/lsu-wine-env:78:in `<main>'
/usr/local/steam-utils/lxbin/../bin/lsu-wine-env:45:in `exec_with_wine_env': unhandled exception
shkhln commented 3 years ago

I get this when I am starting a game, am I missing something?

Something is wrong with the 32-bit Wine part.

ericbsd commented 3 years ago

I get this when I am starting a game, am I missing something?

Something is wrong with the 32-bit Wine part.

ericsteam@ericbsd-ghostbsd-pc:~ % env | grep LSU
LSU_i386_PKG_ROOT=/home/ericsteam/.i386-wine-pkg
shkhln commented 3 years ago

Something is wrong with the 32-bit Wine part.

Probably a conflict with i386-wine(-devel). Completely forgot about that one.

ericbsd commented 3 years ago

Oh possible i will remove it and see how it goes.

ericbsd commented 3 years ago

looks to work now it is installing mono

darkfiberiru commented 3 years ago

the 32bit is only in the i386 tree. Could a slave port be done to build both in amd64 tree?

It's different to them then then i386-wine(-devel)?

shkhln commented 3 years ago

It's different to them then then i386-wine(-devel)?

You should ask "them", whatever "them" is.

i386-wine doesn't build anything in an amd64 environment, it reuses i386 builds in a very roundabout way. In fact, there is no runtime difference between i386-wine and the i386 pkg trick I've described above, it's merely a streamlined version of the same workaround.

darkfiberiru commented 3 years ago

Thanks for the info and appreciate all the hard work.

shkhln commented 3 years ago

looks to work now it is installing mono

@ericbsd Personally, I always hit "cancel" in that dialog. Does anything else work for you?

ericbsd commented 3 years ago

looks to work now it is installing mono

@ericbsd Personally, I always hit "cancel" in that dialog. Does anything else work for you?

Yes, it is pretty much hit and miss for games, but it adds more games to the mix.

h-2 commented 3 years ago

There is now native Proton support. This requires 3 separate things:

I followed these instructions, but lsu-register-proton fails:

% lsu-register-proton
Can't find the Proton 5.13 directory!
Try `steam "steam://install/1420170"` if it's not installed.
% steam "steam://install/1420170"
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
% lsu-register-proton
Traceback (most recent call last):
        2: from /usr/local/bin/lsu-register-proton:47:in `<main>'
        1: from /usr/local/bin/lsu-register-proton:37:in `find_steamapp_dir'
/usr/local/bin/lsu-register-proton:37:in `read': No such file or directory @ rb_sysopen - /home/hannes/.steam/steam/steamapps/libraryfolders.vdf (Errno::ENOENT)

The second command is actually still successful (steam downloads proton and it appears in successful downlods). But registering still fails.

edit: can't the port be made to include/fetch everything that is needed?

shkhln commented 3 years ago 
/usr/local/bin/lsu-register-proton:37:in `read': No such file or directory @ rb_sysopen - /home/hannes/.steam/steam/steamapps/libraryfolders.vdf (Errno::ENOENT)

Are you trying to directly mount and access the game library from a Windows installation?

The second command is actually still successful (steam downloads proton and it appears in successful downlods). But registering still fails.

Where exactly did you install it?

edit: can't the port be made to include/fetch everything that is needed?

Steam doesn't have any API for reporting download state.

h-2 commented 3 years ago

Thanks for the quick reply!

For some reason it now worked:

% lsu-register-proton
Found Proton 5.13 at /home/hannes2/.steam/steam/steamapps/common/Proton 5.13
Copying files from Proton 5.13...
Found Steam Linux Runtime at /home/hannes2/.steam/steam/steamapps/common/SteamLinuxRuntime_soldier
Copying files from Steam Runtime...
Creating symlinks...
Registering emulators/wine-proton as a compatibility tool...
Done

But the game I tried (Dvinity Original Sin 2) doesn't start up:

GameAction [AppID 2906636616, ActionID 5] : LaunchApp changed task to SiteLicenseSeatCheckout with ""
GameAction [AppID 2906636616, ActionID 5] : LaunchApp changed task to CreatingProcess with ""
GameAction [AppID 2906636616, ActionID 5] : LaunchApp waiting for user response to CreatingProcess ""
GameAction [AppID 2906636616, ActionID 5] : LaunchApp continues with user response "CreatingProcess"
Game update: AppID 0 "EoCApp.exe", ProcID 27742, IP 0.0.0.0:0
GameAction [AppID 2906636616, ActionID 5] : LaunchApp changed task to Completed with ""
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
>>> Adding process 27742 for game ID 33554432
>>> Adding process 27743 for game ID 33554432
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
pid 27746 != 27743, skipping destruction (fork without exec?)

(steam:27405): Gtk-WARNING **: 23:32:42.524: gtk_disable_setlocale() must be called before gtk_init()

(steam:27405): GLib-CRITICAL **: 23:32:42.525: unquote_string_inplace: assertion 'err == NULL || *err == NULL' failed
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
>>> Adding process 27744 for game ID 33554432
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.

(steam:27405): Gtk-WARNING **: 23:32:42.572: gtk_disable_setlocale() must be called before gtk_init()

(steam:27405): GLib-CRITICAL **: 23:32:42.572: unquote_string_inplace: assertion 'err == NULL || *err == NULL' failed
ERROR: ld.so: object 'steamfix.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/home/hannes2/.steam/steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
>>> Adding process 27745 for game ID 33554432
>>> Adding process 27747 for game ID 33554432
pressure-vessel-wrap[27743]: E: Cannot run bwrap: wait status 256
pressure-vessel-wrap[27743]: E: Diagnostic output:
bwrap: prctl(PR_SET_NO_NEW_CAPS) failed: Invalid argument

>>> Adding process 27748 for game ID 33554432
Game removed: AppID 0 "EoCApp.exe", ProcID 27742 
Uploaded AppInterfaceStats to Steam
>>> Adding process 27749 for game ID 33554432

I manually selected this:

image

Was that correct?

edit: In my previous post, I ran the last command as the wrong user. Sorry for the confusion.

shkhln commented 3 years ago

Was that correct?

Restart Steam and select emulators/wine-proton.

h-2 commented 3 years ago

Yep, sorry for acting stupid with this. My window manager doesn't have a system tray and I don't use steam frequently. I thought that I had restarted steam, but it was just "minimised".

Anyway, now I have emulators/wine as option and I selected it. But starting still fails:

nv-sglrun /usr/local/steam-utils/lxbin/../bin/lsu-wine-env /usr/home/hannes2/.steam/steam/compatibilitytools.d/FreeBSD_Proton/proton/proton waitforexitandrun /games/DOS2/DefEd/bin/EoCApp.exe
/usr/local/steam-utils/bin/../lxbin/lsu-proton: line 28: exec: nv-sglrun: not found

I googled this and found emulators/libc6-shim which I installed. Then I get

shim init
env: python3: No such file or directory

On FreeBSD Python is not available without the full version. Because this is annoying, I had a symlink from python3.8 -> ~/bin/python3 but this doesn't seem to help. I would have changed the shbang but I don't know which script is actually failing. The things installed by libc6-shim seem to be Ruby scripts.

edit:

I have change /usr/home/hannes2/.steam/steam/compatibilitytools.d/FreeBSD_Proton/proton/proton to call python3.8 directly. Now wine starts up and wants to install Mono to which I agreed.

I then get this log:

....
007c:warn:module:load_so_dll failed to load .so lib L"\\??\\C:\\windows\\system32\\c_949.nls"
007c:warn:module:load_dll Failed to load module L"C:\\windows\\system32\\c_949.nls"; status=c000012f
007c:warn:module:dlopen_dll failed to load .so lib "/usr/home/hannes2/.steam/steam/steamapps/compatdata/2906636616/pfx/dosdevices/z:/usr/home/hannes2/.i386-wine-pkg/usr/local/wine-proton/bin/../share/wine/nls/c_950.nls": /usr/home/hannes2/.steam/steam/steamapps/compatdata/2906636616/pfx/dosdevices/z:/usr/home/hannes2/.i386-wine-pkg/usr/local/wine-proton/bin/../share/wine/nls/c_950.nls: invalid file format
007c:warn:module:load_so_dll failed to load .so lib L"\\??\\Z:\\usr\\home\\hannes2\\.i386-wine-pkg\\usr\\local\\wine-proton\\bin\\..\\share\\wine\\nls\\c_950.nls"
007c:warn:module:load_dll Failed to load module L"\\\\?\\Z:\\usr\\home\\hannes2\\.i386-wine-pkg\\usr\\local\\wine-proton\\bin\\..\\share\\wine\\nls\\c_950.nls"; status=c000012f
007c:warn:module:dlopen_dll failed to load .so lib "/usr/home/hannes2/.steam/steam/steamapps/compatdata/2906636616/pfx/dosdevices/c:/windows/syswow64/c_950.nls": /usr/home/hannes2/.steam/steam/steamapps/compatdata/2906636616/pfx/dosdevices/c:/windows/syswow64/c_950.nls: invalid file format
...
...
007c:err:setupapi:create_dest_file failed to create L"C:\\windows\\system32\\msxml.dll" (error=80)
007c:err:setupapi:create_dest_file failed to create L"C:\\windows\\system32\\msxml2.dll" (error=80)
...
...
wine: configuration in L"/usr/home/hannes2/.steam/steam/steamapps/compatdata/2906636616/pfx" has been updated.
shim init
0024:trace:steam:main 
RecordSteamInterfaceCreation (PID 2977): SteamUtils009 / Utils
[S_API FAIL] SteamAPI_Init() failed; no appID found.
Either launch the game from Steam, or put the file steam_appid.txt containing the correct appID in your game folder.
0024:err:steam:setup_steam_registry SteamAPI_Init failed
0024:trace:steam:convert_linux_vrpaths openvr vrpaths is empty
0024:trace:steam:run_process Converting unix command: L"/games/DOS2/DefEd/bin/EoCApp.exe"
0024:trace:steam:run_process Running command L"\"Z:\\games\\DOS2\\DefEd\\bin\\EoCApp.exe\""
0024:trace:steam:run_process Executing via "CreateProcessW"
0090:fixme:fsync:do_fsync futexes not supported on this platform.
0090:warn:module:load_builtin_dll cannot find builtin library for L"osiris_x64.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"PxFoundation_x64.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"SDL2.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"iggy_w64.dll"
0090:warn:module:alloc_module disabling no-exec because of L"iggy_w64.dll"
0090:warn:module:alloc_module disabling no-exec because of L"d3d11.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"PhysX3Common_x64.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"PhysX3Cooking_x64.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"PhysX3_x64.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"bink2w64.dll"
0090:warn:module:alloc_module disabling no-exec because of L"bink2w64.dll"
0090:warn:module:load_builtin_dll cannot find builtin library for L"Galaxy64.dll"
0090:fixme:heap:RtlSetHeapInformation 0000000001BA0000 0 000000000021E730 4 stub
0090:fixme:heap:RtlSetHeapInformation 0000000001DE0000 0 000000000021E730 4 stub
0090:fixme:ntdll:create_logical_proc_info stub
0090:fixme:ntdll:create_logical_proc_info stub
0090:fixme:thread:SetThreadIdealProcessorEx (FFFFFFFFFFFFFFFE 000000000021F918 0000000000000000): stub
0090:fixme:seh:get_thread_times not implemented on this platform
0090:fixme:thread:SetThreadIdealProcessorEx (FFFFFFFFFFFFFFFE 000000000021F8D0 0000000000000000): stub
shim init
00a0:err:service:device_notify_proc failed to open RPC handle, error 1722
00a4:fixme:service:I_ScRegisterDeviceNotification Notification filters are not yet implemented.
shim_fcntl_impl(117, 1033, ...) is not implemented
Assertion failed: (0), function shim_fcntl_impl, file src/libc/fcntl.c, line 136.
shkhln commented 3 years ago

shim_fcntl_impl(117, 1033, ...) is not implemented Assertion failed: (0), function shim_fcntl_impl, file src/libc/fcntl.c, line 136.

Hmm… I no longer own a Kepler GPU; there might be issues with the latest driver, nv-sglrun and older GPUs. Try nv-sglrun vulkaninfo (from graphics/vulkan-tools) and/or nv-sglrun vkquake to verify that it actually works.

h-2 commented 3 years ago

Thanks for your help on this!

nv-sglrun vulkaninfo This prints lots of stuff:

==========
VULKANINFO
==========

Vulkan Instance Version: 1.2.173

Instance Extensions: count = 17
===============================
        VK_EXT_acquire_xlib_display            : extension revision 1
        VK_EXT_debug_report                    : extension revision 9
        VK_EXT_debug_utils                     : extension revision 2
        VK_EXT_direct_mode_display             : extension revision 1
        VK_EXT_display_surface_counter         : extension revision 1
        VK_KHR_device_group_creation           : extension revision 1
        VK_KHR_display                         : extension revision 23
        VK_KHR_external_fence_capabilities     : extension revision 1
        VK_KHR_external_memory_capabilities    : extension revision 1
        VK_KHR_external_semaphore_capabilities : extension revision 1
        VK_KHR_get_display_properties2         : extension revision 1
        VK_KHR_get_physical_device_properties2 : extension revision 2
        VK_KHR_get_surface_capabilities2       : extension revision 1
        VK_KHR_surface                         : extension revision 25
        VK_KHR_surface_protected_capabilities  : extension revision 1
        VK_KHR_xcb_surface                     : extension revision 6
        VK_KHR_xlib_surface                    : extension revision 6

Layers: count = 5
=================
VK_LAYER_NV_optimus (NVIDIA Optimus layer) Vulkan version 1.2.155, layer version 1:
        Layer Extensions: count = 0
        Devices: count = 1
                GPU id = 0 (GeForce RTX 2060 SUPER)
                Layer-Device Extensions: count = 0

Running vkquake works.

edit: Although I thought that we didn't actually have vulkan support on Nvidia-freebsd until the 465 driver which is currently in beta?

shkhln commented 3 years ago

Go figure. Try some game other than DOS2 with emulators/wine-proton.

edit: Although I thought that we didn't actually have vulkan support on Nvidia-freebsd until the 465 driver which is currently in beta?

Well, 465 has native Vulkan support; nv-sglrun loads Linux libraries. I've been using this hack for 3 years.

h-2 commented 3 years ago

Try some game other than DOS2 with emulators/wine-proton.

I tried The Witcher 3, same problem:

shim_fcntl_impl(38, 1033, ...) is not implemented
Assertion failed: (0), function shim_fcntl_impl, file src/libc/fcntl.c, line 136.
shkhln commented 3 years ago

I see. A few things you can try (in no particular order):

  1. Grab https://github.com/shkhln/libc6-shim/commit/3c51da5fe3697da581751d07da0dd32b412e1f10 to investigate this a bit further. You'll need to set the LSU_SHIM_WRAPPER environment variable to an absolute path to the nv-sglrun script in your local repo.
  2. Stub F_ADD_SEALS, see what happens.
  3. Update nvidia-driver to 465 (with https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254668 or https://github.com/shkhln/revird-aidivn) and set LSU_SHIM_WRAPPER=with-glibc-shim, that skips the Nvidia-specific part. (We still need the shim to load a few proprietary Steam libraries.)