LizardByte / Sunshine

Self-hosted game stream host for Moonlight.
http://app.lizardbyte.dev/Sunshine/
GNU General Public License v3.0
17.57k stars 850 forks source link

GL Error when trying to stream virtual monitor #2044

Open SzczurekYT opened 7 months ago

SzczurekYT commented 7 months ago

Is there an existing issue for this?

Is your issue described in the documentation?

Is your issue present in the nightly release?

Describe the Bug

Hello I'm creating a second virtual monitor with vkms and trying to stream it with sunshine. But when Moonlight client connects you see a black screen and one error line is spammed in the logs. [2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]

Expected Behavior

Working stream displaying a virtual monitor.

Additional Context

I'm using latest nightly to incorporate latest fix without which the virtual monitor wasn't even detected properly. https://github.com/LizardByte/Sunshine/pull/2030 I'm on wayland.

Side question: The Capture Method (Linux Only) fields lists PipeWire as a capture method but I don't see any mention of it in the docs. Does sunshine have or has plans on supporting pipewire capture? I think that could be useful and a nice way to get another capture method on wayland.

Host Operating System

Linux

Operating System Version

Nobara Linux 39 (fedora fork)

Architecture

64 bit

Sunshine commit or version

a10ec3a98ac6a3f5618dc53469f3c5b48dab36ee (Latest nightly as of now)

Package

Linux - flatpak

GPU Type

AMD

GPU Model

AMD Radeon RX 5600 XT

GPU Driver/Mesa Version

Mesa 23.3.3

Capture Method (Linux Only)

KMS

Config

sunshine_name = Szczurek PC

Apps

N\A

Relevant log output

[2024:01:20:14:04:07]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:01:20:14:04:07]: Info: 
[2024:01:20:14:04:07]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:01:20:14:04:07]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
[2024:01:20:14:04:07]: Info: Executing [Desktop]
[2024:01:20:14:04:07]: Info: CLIENT CONNECTED
[2024:01:20:14:04:07]: Warning: No render device name for: /dev/dri/card0
[2024:01:20:14:04:07]: Info: Screencasting with KMS
[2024:01:20:14:04:07]: Warning: No render device name for: /dev/dri/card0
[2024:01:20:14:04:07]: Info: Found monitor for DRM screencasting
[2024:01:20:14:04:07]: Info: Found connector ID [38]
[2024:01:20:14:04:07]: Info: Found cursor plane [34]
libva info: VA-API version 1.18.0
Xlib:  extension "DRI2" missing on display ":0".
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/vkms_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/vkms_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/vkms_drv_video.so
libva info: va_openDriver() returns -1
[2024:01:20:14:04:07]: Error: Couldn't initialize va display: unknown libva error
[2024:01:20:14:04:07]: Warning: Monitor 1 doesn't support hardware encoding. Reverting back to GPU -> RAM -> GPU
[2024:01:20:14:04:07]: Warning: No render device name for: /dev/dri/card0
[2024:01:20:14:04:07]: Info: Found monitor for DRM screencasting
[2024:01:20:14:04:07]: Info: Found connector ID [38]
[2024:01:20:14:04:07]: Info: Found cursor plane [34]
MESA-LOADER: failed to open vkms: /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/vkms_dri.so: nie można otworzyć pliku obiektu dzielonego: Nie ma takiego pliku ani katalogu (search paths /usr/lib/x86_64-linux-gnu/GL/default/lib/dri, suffix _dri)
[2024:01:20:14:04:07]: Info: SDR color coding [Rec. 709]
[2024:01:20:14:04:07]: Info: Color depth: 8-bit
[2024:01:20:14:04:07]: Info: Color range: [MPEG]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
libva info: VA-API version 1.18.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/radeonsi_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_18
libva info: va_openDriver() returns 0
[2024:01:20:14:04:07]: Error: [hevc_vaapi @ 0x7faffc227100] No usable encoding entrypoint found for profile VAProfileHEVCMain (17).
[2024:01:20:14:04:07]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Nie zaimplementowana funkcja
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
libva info: VA-API version 1.18.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/radeonsi_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_18
libva info: va_openDriver() returns 0
[2024:01:20:14:04:07]: Warning: [hevc_vaapi @ 0x7faffcd63780] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:07]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
[2024:01:20:14:04:08]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
cgutman commented 7 months ago

Please run Sunshine with the MESA_DEBUG=1 environment variable per https://docs.mesa3d.org/debugging.html and post the updated log.

SzczurekYT commented 7 months ago

Here you go sunshine.log

cgutman commented 7 months ago

Hmm, out of memory is pretty strange...

Mesa: error: GL_OUT_OF_MEMORY in glGetTexImage
[2024:01:20:16:49:19]: Error: GL: /run/build/sunshine/src/platform/linux/graphics.cpp:562: [00000505]
cgutman commented 7 months ago

Can you try with Sunshine's min log level set to Debug?

If you can give me steps to create a VKMS setup similar to yours, I can try it here. I have used AMDGPU's VKMS support and it worked fine for me after #2030.

SzczurekYT commented 7 months ago

I just do sudo modprobe vkms and use correct number in sunshine. I also set some resolution in the system settings I think. Never heard of amdgpu vkms though. For the log level I'll try when I will have some time, maybe tomorrow.

SzczurekYT commented 7 months ago

Can you try with Sunshine's min log level set to Debug?

sunshine.log

Also: Where does flatpak sunshine store it's config? Sometimes I would like to edit something manually but I just can't find it. It's not in ~/.config/sunshine and ~/.var/app/dev.lizardbyte.sunshine/config/sunshine doesn't seem to be used.

cgutman commented 7 months ago

OK, I dug into this quite a while. This will not be easy to resolve properly.

The problem turns out to be that the dmabufs used as frame buffers on the vkms device (at least with Mutter) are imported from the rendering GPU. If you force the compositor to run using llvmpipe instead of a hardware accelerated GPU, I'll bet it works fine.

You can see this in debugfs:

cat /sys/kernel/debug/dri/0/state
plane[31]: plane-0
    crtc=crtc-0
    fb=48
        allocated by = gnome-shell
        refcount=3
        format=XR24 little-endian (0x34325258)
        modifier=0x0
        size=1024x768
        layers:
            size[0]=1024x768
            pitch[0]=4096
            offset[0]=0
            obj[0]:
                name=0
                refcount=2
                start=00147000
                size=3145728
                imported=yes <-------- HERE
    crtc-pos=1024x768+0+0
    src-pos=1024.000000x768.000000+0.000000+0.000000
    rotation=1
    normalized-zpos=0
    color-encoding=ITU-R BT.601 YCbCr
    color-range=YCbCr limited range

This is a problem because the kernel explicitly forbids mapping of imported dmabufs

kms_swrast needs to map the buffer to read/copy it, which fails here:

(gdb) bt
#0  kms_sw_displaytarget_map (ws=..., dt=..., flags=...) at ../src/gallium/winsys/sw/kms-dri/kms_dri_sw_winsys.c:272
#1  0x00007fffae198379 in llvmpipe_resource_map (resource=..., level=..., layer=..., tex_usage=...) at ../src/gallium/drivers/llvmpipe/lp_texture.c:523
#2  0x00007fffae198710 in llvmpipe_transfer_map_ms (pipe=..., resource=..., level=..., usage=..., sample=sample@entry=..., box=..., transfer=...) at ../src/gallium/drivers/llvmpipe/lp_texture.c:777
#3  0x00007fffae198769 in llvmpipe_transfer_map (pipe=..., resource=..., level=..., usage=..., box=..., transfer=...) at ../src/gallium/drivers/llvmpipe/lp_texture.c:806
#4  0x00007fffadbf05fe in pipe_texture_map_3d (transfer=..., d=..., h=..., w=..., z=..., y=..., x=..., access=..., level=..., resource=..., context=...) at ../src/gallium/auxiliary/util/u_inlines.h:577
#5  st_texture_image_map (st=st@entry=..., stImage=stImage@entry=..., usage=usage@entry=..., x=x@entry=..., y=y@entry=..., z=z@entry=..., w=..., h=..., d=..., transfer=...) at ../src/mesa/state_tracker/st_texture.c:315
#6  0x00007fffadbc1dee in st_MapTextureImage (ctx=ctx@entry=..., texImage=texImage@entry=..., slice=..., x=x@entry=..., y=y@entry=..., w=w@entry=..., h=..., mode=..., mapOut=..., rowStrideOut=...)
    at ../src/mesa/state_tracker/st_cb_texture.c:547
#7  0x00007fffade141bf in get_tex_rgba_uncompressed (transferOps=..., texImage=..., pixels=..., type=..., format=..., depth=..., height=..., width=..., zoffset=..., yoffset=..., xoffset=..., dimensions=..., ctx=...)
    at ../src/mesa/main/texgetimage.c:478
#8  get_tex_rgba (texImage=..., pixels=..., type=..., format=..., depth=..., height=..., width=..., zoffset=..., yoffset=..., xoffset=..., dimensions=..., ctx=...) at ../src/mesa/main/texgetimage.c:605
#9  _mesa_GetTexSubImage_sw (ctx=ctx@entry=..., xoffset=xoffset@entry=..., yoffset=..., yoffset@entry=..., zoffset=..., zoffset@entry=..., width=width@entry=..., height=..., 
    height@entry=..., depth=..., format=..., type=..., pixels=..., texImage=...) at ../src/mesa/main/texgetimage.c:760
#10 0x00007fffadbc4b85 in st_GetTexSubImage
    (ctx=ctx@entry=..., xoffset=xoffset@entry=..., yoffset=yoffset@entry=..., zoffset=zoffset@entry=..., width=width@entry=..., height=height@entry=..., depth=..., format=..., type=..., pixels=..., texImage=...)
    at ../src/mesa/state_tracker/st_cb_texture.c:2733
#11 0x00007fffade13539 in get_texture_image
    (ctx=ctx@entry=..., texObj=texObj@entry=..., target=..., level=level@entry=..., xoffset=xoffset@entry=..., yoffset=yoffset@entry=..., zoffset=..., width=..., height=..., depth=..., format=..., type=..., pixels=..., caller=...) at ../src/mesa/main/texgetimage.c:1441
#12 0x00007fffade1515a in _mesa_GetTextureSubImage (texture=..., level=..., xoffset=..., yoffset=..., zoffset=..., width=..., height=..., depth=..., format=..., type=..., bufSize=..., pixels=...)
    at ../src/mesa/main/texgetimage.c:1633
#13 0x00000000006620d8 in platf::kms::display_ram_t::snapshot(std::function<bool (std::shared_ptr<platf::img_t>&)> const&, std::shared_ptr<platf::img_t>&, std::chrono::duration<long, std::ratio<1l, 1000l> >, bool)
    (this=..., pull_free_image_cb=..., img_out=..., timeout=..., cursor=...) at /home/cgutman/moonlight-src/LB_Sunshine/src/platform/linux/kmsgrab.cpp:1267
#14 0x000000000066178f in platf::kms::display_ram_t::capture(std::function<bool (std::shared_ptr<platf::img_t>&&, bool)> const&, std::function<bool (std::shared_ptr<platf::img_t>&)> const&, bool*)
    (this=..., push_captured_image_cb=..., pull_free_image_cb=..., cursor=...) at /home/cgutman/moonlight-src/LB_Sunshine/src/platform/linux/kmsgrab.cpp:1162
#15 0x00000000005dd928 in video::captureThread (capture_ctx_queue=..., display_wp=..., reinit_event=..., encoder=...) at /home/cgutman/moonlight-src/LB_Sunshine/src/video.cpp:1299

Fortunately this only applies to the swrast codepath that is used when performing CPU encoding or readback VAAPI encoding. It is possible to avoid this issue when performing GPU encoding assuming the primary GPU can import the FB from VKMS.

LizardByte-bot commented 4 months ago

It seems this issue hasn't had any activity in the past 90 days. If it's still something you'd like addressed, please let us know by leaving a comment. Otherwise, to help keep our backlog tidy, we'll be closing this issue in 10 days. Thanks!

d-air1 commented 4 months ago

Keep alive.

LizardByte-bot commented 1 month ago

It seems this issue hasn't had any activity in the past 90 days. If it's still something you'd like addressed, please let us know by leaving a comment. Otherwise, to help keep our backlog tidy, we'll be closing this issue in 10 days. Thanks!

SzczurekYT commented 1 month ago

Nope, still not resolved, don't mark it as stale. @cgutman Do you still plan on finishing your pr #2053 ?