xdg-desktop-portal-wlr previously only set the buffer type of the first plane of the buffer. This, among possibly other issues, causes the file descriptor of DMA-BUF buffers to get lost somewhere between the pipewire daemon and the consuming application (e.g. obs).
The client libpipewire will see a buffer with a type of -1, and will proceed to drop the file descriptor, leaving the user with an invalid DMA-BUF if the format has more than one plane. This fixes that issue.
I initially encountered this issue trying to add xdg-desktop-portal support to wl-mirror, but the same issue also happens with obs-studio.
With obs, this manifests as obs first negotiating a tiled format with 2 planes, failing to import the broken DMA-BUF, and then renegotiating to a different format with only one plane. After this patch, obs successfully imports the initial DMA-BUF with 2 planes.
columbarius
commented
1 year ago
xdg-desktop-portal-wlr previously only set the buffer type of the first plane of the buffer. This, among possibly other issues, causes the file descriptor of DMA-BUF buffers to get lost somewhere between the pipewire daemon and the consuming application (e.g. obs).
The client libpipewire will see a buffer with a type of -1, and will proceed to drop the file descriptor, leaving the user with an invalid DMA-BUF if the format has more than one plane. This fixes that issue.
I initially encountered this issue trying to add xdg-desktop-portal support to wl-mirror, but the same issue also happens with obs-studio.
With obs, this manifests as obs first negotiating a tiled format with 2 planes, failing to import the broken DMA-BUF, and then renegotiating to a different format with only one plane. After this patch, obs successfully imports the initial DMA-BUF with 2 planes.
This can be tested easily by simply adding a debug print into obs-studio/plugins/linux-pipewire/pipewire.c printing out the fd and type of the buffer.