search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.22k stars 174 forks source link

Steam In-Home Streaming is slow with Vulkan-based games #5591

Closed urbenlegend closed 1 year ago

urbenlegend commented 6 years ago

Your system information

System Info

Please describe your issue in as much detail as possible:

Whenever I try to stream Vulkan games to my Steam Link I get considerably lower FPS.

Case 1: Rise of the Tomb Raider Enabling the streaming performance overlay, I can see that it shows "Slow Capture". FPS is borderline unplayable at 20-ish FPS.

Case 2: Mad Max I tried both OpenGL and Vulkan with this game. Without in-home streaming, Vulkan vastly outperforms OpenGL. However, when I try to stream the game in Vulkan mode, Steam Link reports a 20 FPS drop and the gameplay stutters quite a bit. Streaming with OpenGL mode seems relatively normal and smoother, even though its inherently slower than Vulkan.

All my other games are OpenGL based and they run great with Steam Link.

Steps for reproducing this issue:

  1. Run either Rise of the Tomb Raider or Mad Max in Vulkan mode
  2. Stream to Steam Link
  3. Notice stuttering and FPS drops
lewiji commented 2 years ago

@kisak-valve @Plagman apologies for randomly tagging, I'm just (desperately) trying to find someone at Valve who will show an interest or can find help.

Steam Link/Remote Play from a Linux host has been unbearably laggy on many (most?) titles for a couple of years now, and the probable cause has just been identified above - unnecessarily high bit depth used in the Vulkan capture/streaming pipeline when compared to the OpenGL capture - likely by mistake - causes a severe bottleneck in the streaming bandwidth, which limits frame rate, causes intermittent lag spikes, connection issues, and results in huge input lag.

Any idea who would be a relevant person to look into this at Valve?

klepp0906 commented 2 years ago

@kisak-valve @Plagman apologies for randomly tagging, I'm just (desperately) trying to find someone at Valve who will show an interest or can find help.

Steam Link/Remote Play from a Linux host has been unbearably laggy on many (most?) titles for a couple of years now, and the probable cause has just been identified above - unnecessarily high bit depth used in the Vulkan capture/streaming pipeline when compared to the OpenGL capture - likely by mistake - causes a severe bottleneck in the streaming bandwidth, which limits frame rate, causes intermittent lag spikes, connection issues, and results in huge input lag.

Any idea who would be a relevant person to look into this at Valve?

not limited to linux hosts. have had the same issue on windows hosts since ever. been using a workaround for a long time to get vulkan to do anything but choke.

(just in case an employee does pop in and show interest)

urmamasllama commented 2 years ago

So in preparation for receiving my steam deck that is currently in the mail I've been looking into other issue with remote play. I finally tackled fixing BPM working for steamlink which requires launching steam with either -pipewire or -pipewire-dmabuf. there is also a slightly more janky workaround someone had using gamescope. but these got me thinking about our issue. what if we could trick the remote play capture into capturing the whole xwayland or possibly gamescope instance instead of directly capturing the game? In theory this would then be using the opengl capture logic and bypass the issue. I'm not quite sure how to pull it off but I'm going to play around with using pipewire and gamescope on games to see if I can make something happen.

urmamasllama commented 2 years ago

Holy crap it worked. I did some testing on my lunch break. I launched steam with steam -pipewire-dmabuf (regular pipewire has poor capture performance) then I connected to host with the steamlink app (phone, flatpak, physical steamlink)(regular remote play wouldn't work it fails to connect) launch vulkan mode valheim using gamescope launch options and it freaking worked proof note I had some input issues but I ran out of lunchbreak time to test.

Please note for this to work you will need to have a Wayland DE, and pipewire for your audio.

insanemal commented 2 years ago

Does pipewire-dmabuf not function under X11?

That is a shame. But yes this appears to avoid the CPU heavy colour space translation

On Sat, 27 Aug 2022, 2:44 am Ben Massingill, @.***> wrote:

Holy crap it worked. I did some testing on my lunch break. I launched steam with steam -pipewire-dmabuf (regular pipewire has poor capture performance) then I connected to host with the steamlink app (phone, flatpak, physical steamlink)(regular remote play wouldn't work it fails to connect) launch valheim using gamescope launch options and it freaking worked [image: proof] https://user-images.githubusercontent.com/431715/186952631-0fdc25c7-e36f-49fd-a414-5333f4183738.png note I had some input issues but I ran out of lunchbreak time to test.

Please note for this to work you will need to have a Wayland DE, and pipewire for your audio.

— Reply to this email directly, view it on GitHub https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1228711495, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABUQFVUHPMTE4DORTIIDZ3V3DYABANCNFSM4FMJ5SNQ . You are receiving this because you commented.Message ID: @.***>

urmamasllama commented 2 years ago

@insanemal feel free to try but at the very least you will need to have pipewire for your audio system. AFAIK the desktop capture stuff it does only works in wayland but I could be wrong.

edit: thinking about it more you might be able to get it to work with just gamescope. no reason to not give it a try. with wayland pipewire is required because you can't connect to the host BPM in steamlink without it.

VoodaGod commented 2 years ago

@urmamasllama for me using "steam -pipewire-dmabuf" under plasma wayland unfortunately still captures the gamescope window with "Game Vulkan RGB + libyuv" with the same bad performance, even though the desktop is captured with "Desktop PipeWire DMABUF". as before, disabling the steam overlay enables desktop capture with great performance, but you lose a lot of features that way

how are you launching the games with gamescope? have you tried other games?

urmamasllama commented 2 years ago

unfortunately while it worked to give me direct features. Steam input straight up wasn't working and I could never get it to work with the steam overlay trick.

On Sat, Aug 27, 2022 at 1:33 PM Jason Rensburger @.***> wrote:

@urmamasllama https://github.com/urmamasllama for me using "steam -pipewire-dmabuf" under plasma wayland unfortunately still captures the gamescope window with "Game Vulkan RGB + libyuv" with the same bad performance, even though the desktop is captured with "Desktop PipeWire DMABUF". as before, disabling the steam overlay enables desktop capture with great performance, but you lose a lot of features that way

how are you launching the games with gamescope? have you tried other games?

— Reply to this email directly, view it on GitHub https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1229218810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJMYYK7CGSTPV4RXFLMATV3I33TANCNFSM4FMJ5SNQ . You are receiving this because you were mentioned.Message ID: @.***>

urmamasllama commented 2 years ago

I stand corrected I guess the problem was I hadn't solved using the steam link app yet. disabling overlay does work for me no need for gamescope. I do agree though that, is a lot of missing functionality.

On Sat, Aug 27, 2022 at 1:33 PM Jason Rensburger @.***> wrote:

@urmamasllama https://github.com/urmamasllama for me using "steam -pipewire-dmabuf" under plasma wayland unfortunately still captures the gamescope window with "Game Vulkan RGB + libyuv" with the same bad performance, even though the desktop is captured with "Desktop PipeWire DMABUF". as before, disabling the steam overlay enables desktop capture with great performance, but you lose a lot of features that way

how are you launching the games with gamescope? have you tried other games?

— Reply to this email directly, view it on GitHub https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1229218810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJMYYK7CGSTPV4RXFLMATV3I33TANCNFSM4FMJ5SNQ . You are receiving this because you were mentioned.Message ID: @.***>

lewiji commented 2 years ago

Disabling steam overlay isn't an option for me as I use a steam controller. And, I'm on nvidia so gamescope barely works

DistantThunder commented 2 years ago

Spec.

This is still a problem to this day on Wayland for 2 years now, I haven't gotten the level of functionality I had on X11 for In-Home Streaming with the Samsung TV Steam Link app at least.

The strangest thing is that it actually works on the Steam Deck, which is supposedly running under a Wayland compositor as well. Here on the Desktop :

1662669299110c

steam_debug.webm

urmamasllama commented 2 years ago

It's -pipewire not --pipewire. It's weird I know. If you do it right you get an xdg portal popup. Pick the whole desktop capture option

On Thu, Sep 8, 2022, 6:45 PM Distant Thunder @.***> wrote:

Spec.

  • KDE Plasma 5.25
  • Arch Linux
  • Wayland
  • AMD / Mesa RDNA 2

This is still a problem to this day on Wayland for 2 years now, I haven't gotten the level of functionality I had on X11 for In-Home Streaming with the Samsung TV Steam Link app at least.

The strangest thing is that it actually works on the Steam Deck, which is supposedly running under a Wayland compositor as well. Here on the Desktop :

  • Steam BP mode is still a dark windows when running with "--pipewire".
  • In the same mode, games window render dark as well
  • Nothing on the TV but sound and stats
  • Don't know if related but in BP mode, the cursor now has times when it disappears

[image: 1662669299110c] https://user-images.githubusercontent.com/3923810/189227037-fc056cdc-e910-48ea-b10f-c689344b3235.jpg

steam_debug.webm https://user-images.githubusercontent.com/3923810/189227134-4a412d80-b6a8-4fa7-97c7-39ddf2b85fd0.webm

— Reply to this email directly, view it on GitHub https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1241241675, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJMY7BR7JM6MQBXNDGLC3V5JJQBANCNFSM4FMJ5SNQ . You are receiving this because you were mentioned.Message ID: @.***>

onekopaka commented 2 years ago

After September 13th's Steam Beta Update: https://steamcommunity.com/groups/SteamClientBeta/announcements/detail/3380537195892063927 I am now seeing the encoder on Vulkan games as: Vulkan NV12 + NVENC H264 thereby skipping the problematic conversion in libyuv, and as a result, full framerate remote play of my games.

Thanks for getting this fixed Valve folks!

Saroumane commented 2 years ago

I did some "retest" using :

I see a big improvement over the previous 25ish fps BUT : ~~I never really reach 60 fps on different 2D / 3D games I tried. I stay around 59.7 fps, and that creates micro-stutters.~~ I can reach 60 fps once I change "Framerate limit" from "Auto" to "60 fps" on the Steam Deck in Remote Play options => I guess it should be reported to the Steam Deck issue tracker, but I can't find it.

urmamasllama commented 1 year ago

try setting bandwidth to 40 or 50mbps instead of automatic you are bumping up against the auto bandwidth limit which likes to keep things below 30mbps.

On Mon, Sep 19, 2022 at 7:35 PM Saroumane @.***> wrote:

I did some "retest" using :

  • Latest beta ( Built : Sep 14 2022, at 00:06:33 Steam package versions 1663116601)
  • Host : Ryzen3600X / RX 5700 XT) on up-to-date Ubuntu 22.04 with default server (Wayland)
  • Host encoding : tried both software encoder (8 threads) or VAAPI hardware encoder. Same fps, SW encoder looks better.
  • Client : Steam Deck with settings "Beautiful" and Bandwidth : Automatic
  • Gigabit LAN

I see a big improvement over the previous 25ish fps BUT : I never really reach 60 fps on different 2D / 3D games I tried. I stay around 59.7 fps, and that creates micro-stutters.

[image: image] https://user-images.githubusercontent.com/22261137/191124577-fe1f68bf-c39c-4ff6-a658-390d4e74bcb3.png

— Reply to this email directly, view it on GitHub https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1251614905, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJMY5U35A755YN5BLZM6TV7DPTTANCNFSM4FMJ5SNQ . You are receiving this because you were mentioned.Message ID: @.***>

insanemal commented 1 year ago

I run Arch and am using pipewire already.

And yes it does provide screen capture on X11

But it doesn't appear to use dma-buf under x11. As X11 uses a different mechanism for bugger sharing.

On Sat, 27 Aug 2022, 2:50 am Ben Massingill, @.***> wrote:

@insanemal https://github.com/insanemal feel free to try but at the very least you will need to have pipewire for your audio system. AFAIK the desktop capture stuff it does only works in wayland but I could be wrong.

— Reply to this email directly, view it on GitHub https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1228716230, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABUQFWX6H6Y2MR4JJPDK6TV3DYWBANCNFSM4FMJ5SNQ . You are receiving this because you were mentioned.Message ID: @.***>

DistantThunder commented 1 year ago

Replying to https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1241387475

Thanks. Tried with "-pipewire-dmabuf" and "-pipewire" respectively. I did get the XDG prompt but I still got a black screen on my TV, even in game. That is, for OpenGL games.

Anything Vulkan seems to be OK when manually launched from the computer itself.

For anything else, it's a black screen and always displaying "Desktop Capture OpenGL NV12 xxx".

Noticed this however:

CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted CDesktopCapturePipeWire: Couldn't get image data CDesktopCapturePipeWire: Couldn't import dmabuf: Function not implemented CDesktopCapturePipeWire: Failed to mmap memory: Operation not permitted

Repeated all the way until end of stream. Also the mouse cursor show occasionally.

GitThisBugOff commented 1 year ago

Replying to https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1286876270

I'm also getting this with my system.

Steam doesn't also detect VAAPI encoding despite having the necessary packages installed.

kisak-valve commented 1 year ago

Closing as fixed. Please open separate issue reports for any follow up issues.