Image stopped displaying on Steam OS 3.5 (Steam Deck)

[parkerlreed]

OBS 28.1.2 Flatpak OBS-VKCapture 1.2.2 Flatpak OBS-VKCapture tools 1.2.2 locally compiled

The Game Capture source properly resizes if I resize the window but no image is displayed. Started with fresh scene to make sure I hadn't messed anything up.

[nowrep]

Please post OBS log.

[parkerlreed]

Here you are!

2023-01-03 17-38-44.txt

[parkerlreed]

Aha I see this mentioned in the README. Is this new? Haven't had to use it before. It does start working with the environment variable OBS_VKCAPTURE_LINEAR=1 for both Vulkan and OpenGL

[nowrep]

It's not new, and I'm not sure why it happens - I can't reproduce it on my side.

[nowrep]

Or it's because that specific modifier requires GL_TEXTURE_EXTERNAL_OES.

[nowrep]

Can you confirm #129 fixes it? (without having to set OBS_VKCAPTURE_LINEAR=1).

[parkerlreed]

obs-vkcapture-git-r230.c94c34b

No change it seems (Was only able to rebuild the client and not the plugin, since the plugin is in flatpak, if that matters)

12:51:19.163: [linux-vkcapture] Could not create texture from dmabuf source
12:51:19.212: [linux-vkcapture] Creating texture from dmabuf 742x556 modifier:144115188215757571
12:51:19.212: [linux-vkcapture]  [0] fd:47 stride:3072 offset:0
12:51:19.212: [linux-vkcapture]  [1] fd:48 stride:1024 offset:1966080
12:51:19.212: Cannot create EGLImage: Arguments are inconsistent (for example, a valid context requires buffers not supplied by a valid surface).

[nowrep]

Can you try to build the PR and post the output again?

[parkerlreed]

(130)(deck@steamdeck obs-vkcapture-git)$ obs-gamecapture vkcube
[obs-vkcapture] Init Vulkan 1.2.2
Selected GPU 0: AMD Custom GPU 0405 (RADV VANGOGH), type: IntegratedGpu
[obs-vkcapture] Texture VK_FORMAT_B8G8R8A8_UNORM 500x500
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  0: modifier:144115188215757571 planes:2
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  1: modifier:144115188215741187 planes:3
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  2: modifier:144115188215544579 planes:3
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  3: modifier:144115188214274819 planes:1
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  4: modifier:144115188080056578 planes:1
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  5: modifier:144115188075858177 planes:1
[obs-vkcapture] features 6 types 512 handletypes 512

[obs-vkcapture]  6: modifier:0 planes:1
[obs-vkcapture] ------------------ vulkan capture started ------------------

13:46:13.420: Cannot create EGLImage: Arguments are inconsistent (for example, a valid context requires buffers not supplied by a valid surface).
13:46:13.420: [linux-vkcapture] Could not create texture from dmabuf source
13:46:13.469: [linux-vkcapture] Creating texture from dmabuf 878x409 modifier:144115188215757571
13:46:13.469: [linux-vkcapture]  [0] fd:47 stride:3584 offset:0
13:46:13.469: [linux-vkcapture]  [1] fd:48 stride:1024 offset:1835008
13:46:13.470: Cannot create EGLImage: Arguments are inconsistent (for example, a valid context requires buffers not supplied by a valid surface).
13:46:13.470: [linux-vkcapture] Could not create texture from dmabuf source
13:46:24.386: [linux-vkcapture] Creating texture from dmabuf 0x0 modifier:0
13:46:24.386: Cannot create EGLImage: One or more argument values are invalid.
13:46:24.386: [linux-vkcapture] Could not create texture from dmabuf source
13:46:25.369: [linux-vkcapture] Creating texture from dmabuf 500x500 modifier:144115188215757571
13:46:25.370: [linux-vkcapture]  [0] fd:45 stride:2048 offset:0
13:46:25.370: [linux-vkcapture]  [1] fd:46 stride:512 offset:1048576
13:46:25.370: Cannot create EGLImage: Arguments are inconsistent (for example, a valid context requires buffers not supplied by a valid surface).
13:46:25.370: [linux-vkcapture] Could not create texture from dmabuf source

[parkerlreed]

Did notice some new behavior. OpenGL works fine on the PR. Vulkan still does that

[nowrep]

OpenGL works fine on the PR

That PR doesn't touch the GL parts, so it shouldn't have any effect.

Thanks for the testing, I don't have any more ideas sadly. For now, you'll have to keep using OBS_VKCAPTURE_LINEAR=1 it seems.

[parkerlreed]

Oh weird... I swear OpenGL was also having issues without OBS_VKCAPTURE_LINEAR=1 but just retested on master and it's fine.

Thanks for looking into it!

[nowrep]

This should also work (and likely more optimal than forcing linear):

diff --git a/src/vklayer.c b/src/vklayer.c
index de032b1..12ab952 100644
--- a/src/vklayer.c
+++ b/src/vklayer.c
@@ -1551,6 +1551,8 @@ static VkResult VKAPI_CALL OBS_CreateDevice(VkPhysicalDevice phy_device,
         hlog("DRM format modifier support not available");
     }

+    dfuncs->GetImageDrmFormatModifierPropertiesEXT = NULL;
+
 #undef GETADDR

     if (!funcs_found) {

[parkerlreed]

Indeed it does! Thank you

[nowrep]

Actually another one if you could test please: https://github.com/nowrep/obs-vkcapture/commit/f25de657793751813ae7bbcf09000ccea6fe78c1

[parkerlreed]

r232.f25de65-1

[obs-vkcapture] Init Vulkan 1.2.2
Selected GPU 0: AMD Custom GPU 0405 (RADV VANGOGH), type: IntegratedGpu
[obs-vkcapture] Texture VK_FORMAT_B8G8R8A8_UNORM 500x500
[obs-vkcapture] features 56707

[obs-vkcapture] features 56707

[obs-vkcapture] features 56707

[obs-vkcapture] features 56707

[obs-vkcapture] features 56707

[obs-vkcapture] features 56707

[obs-vkcapture] features 56707

[obs-vkcapture] ------------------ vulkan capture started ------------------

[nowrep]

What about this?

diff --git a/src/vklayer.c b/src/vklayer.c
index 3689dea..b09b1a8 100644
--- a/src/vklayer.c
+++ b/src/vklayer.c
@@ -632,7 +632,7 @@ static inline bool vk_shtex_init_vulkan_tex(struct vk_data *data,
     img_info.extent.width = swap->image_extent.width;
     img_info.extent.height = swap->image_extent.height;
     img_info.extent.depth = 1;
-    img_info.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT;
+    img_info.usage = VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT | VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT;
     img_info.tiling = VK_IMAGE_TILING_LINEAR;

     int num_planes = 1; // format itself may be multi-planar?

[parkerlreed]

Patch performed against modifier-importable and master seemingly no change

[nowrep]

Can you please upload log from OBS: EGL_LOG_LEVEL=debug obs

[nowrep]

In any case, this should fix it (fallback to no modifiers) https://github.com/nowrep/obs-vkcapture/pull/133

[parkerlreed]

Thanks for the updates! Been busy with life, heh

Working great with local OBS but Flatpak is now printing NOTHING to log or the window.

15:52:35.442: User Removed source 'Media Source' (ffmpeg_source) from scene 'Scene 2'
15:52:40.977: User added source 'Game Capture' (vkcapture-source) to scene 'Scene 2'

Flatpak did have some recent stuff with FFMpeg/Gstreamer/VAAPI codecs but I'm not sure if that would affect it at least showing up and processing frames.

[nowrep]

It needs flatpak update because the changes are backwards incompatible.