Cannot create EGLImage: Arguments are inconsistent

[scaledteam]

Hi! OBS Studio 27 now require fourcc in gs_texture_create_from_dmabuf function. I edited code and added fourcc, but it doesn't work. I also tried to set fourcc manually, based on this OBS-Studio deleted code, but i get similar error.

Code:

ctx->texture = gs_texture_create_from_dmabuf(fb->width, fb->height,
    GS_BGRA, // FIXME handle fourcc?
    fb->fourcc,  // i added fourcc there
    1, // FIXME handle planes
    ctx->fbs.fb_fds + index,
    &stride,
    &offset,
    NULL // FIXME what are modifiers? we just don't know

Error:

error: Cannot create EGLImage: Arguments are inconsistent (for example, a valid context requires buffers not supplied by a valid surface).```

[scaledteam]

Sorry, i placed arguments wrong. I drm_format need to be placed before gs_color_format. But this is only part of problem, now i got "EGL failed to allocate resources for the requested operation" error =(((. libobs/graphics/graphics.h:

EXPORT gs_texture_t *gs_texture_create_from_dmabuf(
    unsigned int width, unsigned int height, uint32_t drm_format,
    enum gs_color_format color_format, uint32_t n_planes, const int *fds,
    const uint32_t *strides, const uint32_t *offsets,
    const uint64_t *modifiers);

Error:

error: Cannot create EGLImage: EGL failed to allocate resources for the requested operation.
error: [linux-kmsgrab] Could not create texture from dmabuf source

[w23]

I haven't used this in a while (if you're trying to capture screen on wayland, i'd recommend other plugins, like https://hg.sr.ht/~scoopta/wlrobs). Now that you mention it, I tried to build it and am running into the very same issues.

I'm going to try to gdb mesa and see why it does this to us. (i've also did a bunch of stuff here, in case you feel adventurous: https://github.com/w23/obs-kmsgrab/tree/obs-27-fixes)

[scaledteam]

I use X11 because Wayland let me down so many times. Also, this plugin with yours looks like very good combination: https://github.com/nowrep/obs-vkcapture

[w23]

That's a good reference, thanks!

I still have no clue about the cause. It seems that wlrobs still works for me. And it's only the kmsgrab plugin that broke. Will continue looking.

[scaledteam]

For some reason, i don't have drm/drm_fourcc.h in my system (Debian 11), but have libdrm/drm_fourcc.h. And unfortunally, i get same error, but now fourcc looks lot nicer:

info: [linux-kmsgrab] Received 2 framebuffers:
info: [linux-kmsgrab] Received width=2560 height=1440 planes=1 fourcc=DRM_FORMAT_XRGB8888(0x34325258) fd=41
info: [linux-kmsgrab] Received width=256 height=256 planes=1 fourcc=DRM_FORMAT_ARGB8888(0x34325241) fd=42
error: Cannot create EGLImage: EGL failed to allocate resources for the requested operation.
error: [linux-kmsgrab] Could not create texture from dmabuf source

[w23]

It seems that amdgpu_bo_import fails:

  * frame #0: 0x00007fffec0fb1a0 libdrm_amdgpu.so.1`amdgpu_bo_import
    frame #1: 0x00007fffdd369c6c radeonsi_dri.so`amdgpu_bo_from_handle(rws=<unavailable>, whandle=0x00007fff626a8e50, vm_alignment=65536) at amdgpu_bo.c:1528:8
    frame #2: 0x00007fffdd3128a6 radeonsi_dri.so`si_texture_from_handle(screen=0x00005555559ae520, templ=0x00007fff626a8d30, whandle=0x00007fff626a8e50, usage=2) at si_texture.c:1646:10
    frame #3: 0x00007fffdcb076ec radeonsi_dri.so`dri2_create_image_from_winsys(_screen=0x0000555555dd6e80, width=2560, height=1440, map=<unavailable>, num_handles=3, whandle=0x00007fff626a8df0, is_protected_content=false, loaderPrivate=0x0000000000000000) at dri2.c:807:13
    frame #4: 0x00007fffdcb07e83 radeonsi_dri.so`dri2_create_image_from_fd(_screen=0x0000555555dd6e80, width=2560, height=1440, fourcc=875713089, modifier=144115206334806273, fds=0x00007fff626a8fb0, num_fds=3, strides=0x00007fff626a8fc0, offsets=0x00007fff626a8fd0, protected_content=false, error=0x00007fff626a8fac, loaderPrivate=0x0000000000000000) at dri2.c:968:10
    frame #5: 0x00007fffdcb07f2f radeonsi_dri.so`dri2_from_dma_bufs2(screen=<unavailable>, width=<unavailable>, height=<unavailable>, fourcc=<unavailable>, modifier=<unavailable>, fds=<unavailable>, num_fds=3, strides=0x00007fff626a8fc0, offsets=0x00007fff626a8fd0, yuv_color_space=__DRI_YUV_COLOR_SPACE_UNDEFINED, sample_range=__DRI_YUV_RANGE_UNDEFINED, horizontal_siting=__DRI_YUV_CHROMA_SITING_UNDEFINED, vertical_siting=__DRI_YUV_CHROMA_SITING_UNDEFINED, error=0x00007fff626a8fac, loaderPrivate=0x0000000000000000) at dri2.c:1534:10
    frame #6: 0x00007fffec20ab4d libEGL_mesa.so.0`dri2_create_image_dma_buf(disp=0x0000555555e1cc60, ctx=<unavailable>, buffer=<unavailable>, attr_list=<unavailable>) at egl_dri2.c:2903:10
    frame #7: 0x00007fffec1fa265 libEGL_mesa.so.0`_eglCreateImageCommon(disp=0x0000555555e1cc60, ctx=0x0000000000000000, target=12912, buffer=0x0000000000000000, attr_list=0x00007fff480e2b90) at eglapi.c:1744:10
    frame #8: 0x00007fffec1fa540 libEGL_mesa.so.0`eglCreateImage(dpy=<unavailable>, ctx=0x0000000000000000, target=12912, buffer=0x0000000000000000, attr_list=<unavailable>) at eglapi.c:1774:12
    frame #9: 0x00007fffec075c91 libobs-opengl.so.0`create_dmabuf_egl_image + 705
    frame #10: 0x00007fffec075e79 libobs-opengl.so.0`gl_egl_create_dmabuf_image + 169
    frame #11: 0x00007ffff6107a37 libobs.so.0`gs_texture_create_from_dmabuf + 87
    frame #12: 0x00007fff60f5966f linux-kmsgrab.so`dmabuf_source_open + 626
    frame #13: 0x00007fff60f5975e linux-kmsgrab.so`dmabuf_source_update + 141
    frame #14: 0x00007ffff613c04e libobs.so.0`obs_source_video_tick + 1726
    frame #15: 0x00007ffff6179de3 libobs.so.0`obs_graphics_thread_loop + 387
    frame #16: 0x00007ffff617b3d8 libobs.so.0`obs_graphics_thread + 280
    frame #17: 0x00007ffff525fe5e libpthread.so.0`start_thread + 222
    frame #18: 0x00007ffff5194e1f libc.so.6`__clone + 63

I might not have more time today to dig deeper

[w23]

Uhhh:

info: [linux-kmsgrab] Received width=2560 height=1440 planes=3 fourcc=DRM_FORMAT_ARGB8888(0x34325241) fd=34

...

* thread #35, name = 'libobs: graphic', stop reason = step over
    frame #0: 0x00007ffff29302f4 libdrm.so.2`drmPrimeFDToHandle(fd=<unavailable>, prime_fd=<unavailable>, handle=0x00007fff626a8aa8) at xf86drm.c:2909:8
   2906     memclear(args);
   2907     args.fd = prime_fd;
   2908     ret = drmIoctl(fd, DRM_IOCTL_PRIME_FD_TO_HANDLE, &args);
-> 2909     if (ret)
   2910         return ret;
   2911 
   2912     *handle = args.handle;
(lldb) p ret
(int) $9 = -1
(lldb) p args
(drm_prime_handle) $10 = (handle = 0, flags = 0, fd = 34)

...

  * frame #0: 0x00007ffff29302f4 libdrm.so.2`drmPrimeFDToHandle(fd=<unavailable>, prime_fd=<unavailable>, handle=0x00007fff626a8aa8) at xf86drm.c:2909:8
    frame #1: 0x00007fffec0fc25f libdrm_amdgpu.so.1`amdgpu_bo_import(dev=0x0000555555de1400, type=amdgpu_bo_handle_type_dma_buf_fd, shared_handle=34, output=0x00007fff626a8b30) at amdgpu_bo.c:278:7
    frame #2: 0x00007fffd5369c6c radeonsi_dri.so`amdgpu_bo_from_handle(rws=<unavailable>, whandle=0x00007fff626a8e50, vm_alignment=65536) at amdgpu_bo.c:1528:8
    frame #3: 0x00007fffd53128a6 radeonsi_dri.so`si_texture_from_handle(screen=0x00005555559ae560, templ=0x00007fff626a8d30, whandle=0x00007fff626a8e50, usage=2) at si_texture.c:1646:10
    frame #4: 0x00007fffd4b076ec radeonsi_dri.so`dri2_create_image_from_winsys(_screen=0x0000555555c58fd0, width=2560, height=1440, map=<unavailable>, num_handles=3, whandle=0x00007fff626a8df0, is_protected_content=false, loaderPrivate=0x0000000000000000) at dri2.c:807:13
    frame #5: 0x00007fffd4b07e83 radeonsi_dri.so`dri2_create_image_from_fd(_screen=0x0000555555c58fd0, width=2560, height=1440, fourcc=875713089, modifier=144115206334806273, fds=0x00007fff626a8fb0, num_fds=3, strides=0x00007fff626a8fc0, offsets=0x00007fff626a8fd0, protected_content=false, error=0x00007fff626a8fac, loaderPrivate=0x0000000000000000) at dri2.c:968:10
    frame #6: 0x00007fffd4b07f2f radeonsi_dri.so`dri2_from_dma_bufs2(screen=<unavailable>, width=<unavailable>, height=<unavailable>, fourcc=<unavailable>, modifier=<unavailable>, fds=<unavailable>, num_fds=3, strides=0x00007fff626a8fc0, offsets=0x00007fff626a8fd0, yuv_color_space=__DRI_YUV_COLOR_SPACE_UNDEFINED, sample_range=__DRI_YUV_RANGE_UNDEFINED, horizontal_siting=__DRI_YUV_CHROMA_SITING_UNDEFINED, vertical_siting=__DRI_YUV_CHROMA_SITING_UNDEFINED, error=0x00007fff626a8fac, loaderPrivate=0x0000000000000000) at dri2.c:1534:10
    frame #7: 0x00007fffec20bb4d libEGL_mesa.so.0`dri2_create_image_dma_buf(disp=0x0000555555e1cb60, ctx=<unavailable>, buffer=<unavailable>, attr_list=<unavailable>) at egl_dri2.c:2903:10
    frame #8: 0x00007fffec1fb265 libEGL_mesa.so.0`_eglCreateImageCommon(disp=0x0000555555e1cb60, ctx=0x0000000000000000, target=12912, buffer=0x0000000000000000, attr_list=0x00007fff48260580) at eglapi.c:1744:10
    frame #9: 0x00007fffec1fb540 libEGL_mesa.so.0`eglCreateImage(dpy=<unavailable>, ctx=0x0000000000000000, target=12912, buffer=0x0000000000000000, attr_list=<unavailable>) at eglapi.c:1774:12
    frame #10: 0x00007fffec076c91 libobs-opengl.so.0`create_dmabuf_egl_image + 705
    frame #11: 0x00007fffec076e79 libobs-opengl.so.0`gl_egl_create_dmabuf_image + 169
    frame #12: 0x00007ffff6107a37 libobs.so.0`gs_texture_create_from_dmabuf + 87
    frame #13: 0x00007fff60f5966f linux-kmsgrab.so`dmabuf_source_open + 626
    frame #14: 0x00007fff60f5975e linux-kmsgrab.so`dmabuf_source_update + 141
    frame #15: 0x00007ffff613c04e libobs.so.0`obs_source_video_tick + 1726
    frame #16: 0x00007ffff6179de3 libobs.so.0`obs_graphics_thread_loop + 387
    frame #17: 0x00007ffff617b3d8 libobs.so.0`obs_graphics_thread + 280
    frame #18: 0x00007ffff525fe5e libpthread.so.0`start_thread + 222
    frame #19: 0x00007ffff5194e1f libc.so.6`__clone + 63

So it's the kernel not letting us know about handle (probably because obs itself is not privileged), even though we do have fd.

I also checked that my old obs fork with this stuff built in also stopped working.

yay adventures

[scaledteam]

amdgpu_bo_import

By the way, i use intel videocard for streaming. Also, i remember that few mounths ago your plugin worked fine, but only when OBS launched from root. It was strange beta branch of OBS.

[w23]

It's interesting that we see essentially the same breakage, but for you it's intel + x11, and for me it's sway + amd.

Also, it's weird that you needed to run obs as root. Splitting drmsend binary is supposed to allow obs to run unprivileged.

(btw, running obs privileged does not solve this issue, i've just checked)

[scaledteam]

No, it was few mounths ago. Now it works without root, but with one issue https://github.com/w23/obs-kmsgrab/issues/11

[w23]

So you say that there's some obs revision with which this plugin still works?

For me neither this plugin, nor drmtoy repo (where I experimented with this stuff), nor ffmpeg kmsgrab work anymore. They all seem to fail in drmModeGetFB with invalid argument. drmModeGetFB2 works to retrieve framebuffer data (which has 3 planes/objs and three identical handles/fds). But it still fails tobe imported into EGL...

[scaledteam]

It's good idea to check this. I will check if my old records and backups shows OBS verion later, it will be very lucky if i have some backups of working version.

[w23]

Another day, another session of unproductive head banging against the wall.

I guess next I'm going to bpftrace the shit out of kernel.

[scaledteam]

I found sources of obs 17.0.0-4597-g1d9ab8d3 build with your patches, also some videos from me and one deleted video video from Dmitry Bachilo with 17.0.0-3842-g9b7df516-modified version, where DMA-BUF source was used. Can we contact in Discord?

[scaledteam]

IT WORKS!!!!!

[scaledteam]

Tell me if i forgot something. This is sources of OBS with kmsgrab, that works today. You also need to change backend to EGL in advanced settings, not using OBS_USE_EGL. I usually do ldconfig after every new installation of OBS-Studio, don't know if it's matter in this case. https://scaledteam.ru/p/obs-studio-zero-copy.tar.xz

[scaledteam]

You forgot to send fb_id from drmsend into OBS Studio. I added this into 165 line in drmsend.c, and now fb_id's appears in list, but unfortunately, it doesn't help to solve EGL failed to allocate resources for the requested operation problem. Probably we forgot to send something else?

fb->fb_id = drmfb->fb_id;