[BUG] The video is glitched, looks like checkerboard pattern

[dec05eba]

Describe the bug The video is glitched when recording on wayland

To Reproduce gpu-screen-recorder -w screen -f 60 -o video.mp4

Expected behavior The video shouldn't look glitched

Screenshots

Desktop (please complete the following information):

• X11 or Wayland: Wayland
  • Desktop environment/Window Manager: Gnome
• Distro: Arch Linux
• GPU: Intel Iris Xe Graphics G7
• Version: aur

Additional context Terminal output:

[hevc_vaapi @ 0x6239a97bb380] ignoring invalid SAR: 0/0
[hevc_vaapi @ 0x6239a97bb380] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
Info: using hevc encoder because a codec was not specified
gsr info: gsr_kms_client_init: waiting for server to connect
kms server info: connecting to the client
gsr info: gsr_kms_client_init: server connected
gsr info: replacing file-backed unix domain socket with socketpair
kms server info: connected to the client
gsr info: using socketpair
[hevc_vaapi @ 0x6239a97bb380] ignoring invalid SAR: 0/0
[hevc_vaapi @ 0x6239a97bb380] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
update fps: 247
update fps: 247

drm_info output: drm_info.log

• [x] I played the video with the command mpv --no-config video.mp4 (if applicable)
• [ ] I use a laptop with integrated GPU and dedicated GPU

[dec05eba]

This happens on some intel integrated GPUs on wayland because of display compression which I believe is done for power usage reduction. The only known workaround for this at the moment is to record a window on X11. I dont have the hardware to reproduce this issue (intel xe graphics) so this cant be fixed unless somebody knows how to force I915_FORMAT_MOD_Y_TILED_CCS modifier for the display drm plane on other intel iGPUs or if somebody is willing to donate a device with intel xe graphics.

[guiodic]

I have the same problem on X11. But single-window recording on the other hand is fine.

[guiodic]

gpu-screen-recorder -w screen -f 60 -o video.mp4 [h264_vaapi @ 0x5765a14a9580] ignoring invalid SAR: 0/0 Info: using h264 encoder because a codec was not specified gsr info: gsr_kms_client_init: waiting for server to connect kms server info: connecting to the client gsr info: gsr_kms_client_init: server connected gsr info: replacing file-backed unix domain socket with socketpair kms server info: connected to the client gsr info: using socketpair [h264_vaapi @ 0x5765a14a9580] ignoring invalid SAR: 0/0 update fps: 242 update fps: 243 update fps: 242 update fps: 241 ^C⏎

[guiodic]

other infos: Intel iGPU (Xe Tigerlake) Kernel Linux 6.8.3

[dec05eba]

@guiodic can you run sudo drm_info > log.log on X11 and attach that log.log file here? it's probably the exact same issue but I want to make sure.

[guiodic]

drm_info.log

I tried on Wayland and... it works well! LOL

[dec05eba]

drm_info.log

I tried on Wayland and... it works well! LOL

ok so the issue on x11 is the same as reported here. I guess the issue on wayland depends on which wayland compositor you are running or maybe power settings. Which desktop environment were you running on X11 and did you have any specific power options set? and how about on wayland? also which distro are you running? (it might depend on which version of software you are using, and maybe even kernel version).

[guiodic]

I'm running KDE Plasma 6 on Manjaro Linux (unstable, so pretty the same than Arch) This are i915 paramenters options i915 enable_guc=3 enable_fbc=1 fastboot=1 nuclear_pageflip=1

[dec05eba]

It's that enable_fbc option being set to 1 that breaks recording, so im not sure if its even used for you on wayland. Can you run that drm_info command on wayland as well and attach the log?

[guiodic]

drm_info_wayland.log

[dec05eba]

drm_info_wayland.log

thanks, so yeah that enable_fbc seems to be ignored by your wayland compositor (or the driver? i dont know). It's not compressed. I'll see if its possible to force enable it on my iGPU so I can test it. Thanks, I forgot that enable_fbc option exists.

[guiodic]

Strange because I remember setting this parameter a long time ago but gsr had continued to work on X11

[dec05eba]

its possible that a driver update, mesa or xorg server update enabled it now. It should work if you set enable_fbc=0 instead, but i'll see if i can fix it

[guiodic]

okay but that is a kernel parameter, it should always be active on both Xorg and Wayland, right?

[dec05eba]

Im not sure, it might have to be chosen by the compositor as well. But it also seems like it can be disabled for other reasons, as explained here: https://wiki.archlinux.org/title/intel_graphics#Framebuffer_compression_(enable_fbc)

[dec05eba]

Hmm that enable_fbc=1 has no effect on my intel iGPU so unfortunately I cant reproduce and fix this

[guiodic]

I will try to disable it to see what happens

[quazar-omega]

I can confirm that the same problem happens on my discrete GPU Intel Arc A750

[dec05eba]

I can confirm that the same problem happens on my discrete GPU Intel Arc A750

Oh I didn't know it can happen on discrete GPU too. I have seen it work on intel arc a750 before. Are you on x11 or wayland? If possible can you run sudo drm_info > log.log and upload that file? just to confirm if its caused by the same thing on your gpu. (if drm_info is available in your package manager).

[quazar-omega]

Ah strange, maybe it's an issue on my end then.
I'm using Wayland on KDE.

Here's more of my system's information, if needed:

- Operating System: Fedora Linux 40 - KDE Plasma Version: 6.0.4 - KDE Frameworks Version: 6.2.0 - Qt Version: 6.7.0 - Kernel Version: 6.8.10-300.fc40.x86_64 (64-bit) - Graphics Platform: Wayland - Processors: 16 × AMD Ryzen 7 5700G with Radeon Graphics - Memory: 15,4 GiB of RAM - Graphics Processor: Mesa Intel® Arc - Manufacturer: ASUS

can you run sudo drm_info > log.log

Is it fine if I run it without sudo? Here's the output running unprivileged: drm_info_output.txt I can install it system-wide if you need it of course

[dec05eba]

It needs to be run as root yes. The person that recorded on that gpu might have used x11. Here is the video of that: https://www.youtube.com/watch?v=9gEF-7Y418Y or something has changed in wayland compositors/intel driver since then

[dec05eba]

Also does the output look pretty much the same? or does it glitch in a different way?

[quazar-omega]

It needs to be run as root yes. The person that recorded on that gpu might have used x11. Here is the video of that: youtube.com/watch?v=9gEF-7Y418Y or something has changed in wayland compositors/intel driver since then

Alright, here it is! sudo_drm_info_output.txt

Also does the output look pretty much the same? or does it glitch in a different way?

It's different, yes, it's not completely corrupted, you can still make something out, in the video I move the cursor around and then move a window, here's the video of the video:

https://github.com/dec05eba/gpu-screen-recorder-issues/assets/56311398/d7528f70-b191-4965-b9ab-8ec6539ad686

[dec05eba]

Thanks. Yeah thats a different issue. I have a guess on how to fix it but I dont have an intel arc gpu to test with.

[quazar-omega]

I can test for you if that's possible, do let me know how to go about it.
Btw, since you said it's another thing, should I open a new issue for this? I don't really know how to title it tho, it would have been very similar to this one here if it weren't there lol

[dec05eba]

Btw, since you said it's another thing, should I open a new issue for this

Nah it's fine, it's the same category of issue

[DatCaptainHorse]

Facing same issue, here's the requested info:

Operating System: openSUSE Tumbleweed 20240607
KDE Plasma Version: 6.0.5
KDE Frameworks Version: 6.2.0
Qt Version: 6.7.1
Kernel Version: 6.10.0-rc2-1.g068a181-default (64-bit)
Graphics Platform: Wayland
Graphics Processor: Mesa Intel® Arc (A770 16GB)

drminfo.log

[dec05eba]

@DatCaptainHorse for you its the same issue as @quazar-omega . Can you clone https://repo.dec05eba.com/gpu-screen-recorder apply this patch: patch.txt and run sudo ./install.sh in that repository to install it and then try recording with gpu-screen-recorder -w screen -f 60 -o video.mp4 to see if that fixes the issue?

[quyse]

Hi @dec05eba I have the same issue, glitched recording as in the video in the very first comment. I tried the patch, and it now records black screen with a mouse cursor on it. Mouse cursor is captured without glitches, but there's nothing else, just black background.

I run NixOS 24.05, Sway/Wayland, Intel Iris Graphics. drm_info.txt

[dec05eba]

@quyse can you test if it works with ffmpeg kmsgrab? run sudo setcap cap_sys_admin=ep $(which ffmpeg) and then ffmpeg -device /dev/dri/card0 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,scale_vaapi=w=1920:h=1080:format=nv12' -c:v h264_vaapi -b:v 2500k -maxrate 4000k -y output-file.mkv you can revert the change to the ffmpeg binary afterwards with sudo setcap -r $(which ffmpeg).

[quyse]

@dec05eba Sure, just tried that, it records glitched video similarly to the comment 1, but without mouse pointer. That is with ffmpeg 6.1.1.

[dec05eba]

Ok thanks. Then I cant fix this issue until I get a hold of intel iris graphics myself or until I add support for pipewire capture. The issue that @DatCaptainHorse has is a different one that may be fixable.

[DatCaptainHorse]

@dec05eba No improvements with patch, attached the video incase that's any use.

Since the project is in C/C++ I'll tinker and look around the code for a bit if I notice something.

https://github.com/dec05eba/gpu-screen-recorder-issues/assets/14197772/710e747d-0a86-4cc3-bf84-1d1d70e617d3

[DatCaptainHorse]

@dec05eba The issue is my GPU is using NV12 for the format and I can see you already have a little comment pointing that you don't have support for multi-plane format :sweat_smile:

// TODO: Support other plane formats than rgb (with multiple planes, such as direct YUV420 on wayland).

I'll try my hand at adding the support for atleast NV12.

[dec05eba]

@DatCaptainHorse that's for the input texture, and in your case your gpu uses ABGR2101010 (rgb), not nv12. The issue here has to do with the drm plane modifier, and on your gpu it's set to I915_FORMAT_MOD_4_TILED so the structure of the texture is not stored like a regular texture. But this should be possible to automatically be handled by the driver, amd can do it. Otherwise maybe the shader has to be modified to handle it somehow. In the case of the glitch in the top video it's a different case, in which the texture is compressed.

[DatCaptainHorse]

I did some printing of couple variables if that helps at all to untangle this

drm_fd->pixel_format: 808665665
drm_fd->width: 3840
drm_fd->height: 2160
drm_fd->fd: 15
drm_fd->offset: 0
drm_fd->pitch: 15360
drm_fd->modifier: 72057594037927945

texture_rotation: 0.000000

self->base.color_conversion.params.color_range: 0
self->base.color_conversion.params.source_color: 0
self->base.color_conversion.params.destination_color: 0
self->base.color_conversion.params.load_external_image_shader: 0
self->base.color_conversion.params.num_destination_textures: 2

self->capture_size.x: 3840
self->capture_size.y: 2160
capture_pos.x: 0
capture_pos.y: 0

[dec05eba]

That's the same information that is available in the drm_info output that you provided :) pixel_format 808665665 is the same as DRM_FORMAT_ABGR2101010 and modifier 72057594037927945 is the same as I915_FORMAT_MOD_4_TILED.

[DatCaptainHorse]

Alright, sorry :sweat_smile: - just trying to find something helpful

[dec05eba]

@DatCaptainHorse can you run ffmpeg kmsgrab as described here:

@quyse can you test if it works with ffmpeg kmsgrab? run sudo setcap cap_sys_admin=ep $(which ffmpeg) and then ffmpeg -device /dev/dri/card0 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,scale_vaapi=w=1920:h=1080:format=nv12' -c:v h264_vaapi -b:v 2500k -maxrate 4000k -y output-file.mkv you can revert the change to the ffmpeg binary afterwards with sudo setcap -r $(which ffmpeg).

and post the terminal output here? also verify if the video it outputs looks fine again.

[DatCaptainHorse]

My main system says:

[kmsgrab @ 0x55dccaa28bc0] Using plane 32 to locate framebuffers.
[kmsgrab @ 0x55dccaa28bc0] Template framebuffer is 292: 3840x2160 format 30334241 modifier 100000000000009 flags 2.
[kmsgrab @ 0x55dccaa28bc0] Framebuffer pixel format 30334241 is not a known supported format.
[in#0 @ 0x55dccaa28900] Error opening input: Invalid argument
Error opening input file -.
Error opening input files: Invalid argument

I totally forgot I did it succesfully under Nestri (which uses Xorg), let me apply the earlier patch there.. :sweat_smile:

[dec05eba]

can you run the sudo drm_info > log.log command again but on x11 and upload the file? and the ffmpeg command as well and post the terminal output

[DatCaptainHorse]

Here's the ffmpeg side, let me grab drm_info now that I noticed your message :slightly_smiling_face: ffprintout.txt

Update: Here's drm_info output as well now @dec05eba :+1: drmoutput.txt

[dec05eba]

Ok in this case the drm plane modifier is 0 and the format is XRGB8888. It should work if you record with gpu screen recorder without any modifications. But if it doesn't can you apply that patch and try?

[DatCaptainHorse]

Yeah I tried with the patch, no luck as the output was still artifacty :thinking: - let me try my actual DE in X11 as well, running in containers is time consuming :sweat_smile:

[dec05eba]

Can you take a screenshot (or post the video) of what it looks like with and without the patch? it should look different than what you previously posted I guess?

[DatCaptainHorse]

Top one is without patch applied, bottom one is with patch applied.

https://github.com/dec05eba/gpu-screen-recorder-issues/assets/14197772/c98033ec-36c4-4c12-9983-c1853c2cdbe6

https://github.com/dec05eba/gpu-screen-recorder-issues/assets/14197772/970e3119-8741-4afc-afca-299eaf91ce70

[dec05eba]

hmm that makes no sense. Can you add the printing of variables that you did before? to see if it somehow doesn't match what is in drm_info anymore or what ffmpeg outputs

[dec05eba]

or btw, did you verify again if the ffmpeg produced video looks fine?

[DatCaptainHorse]

Yeah ffmpeg kmsgrab is still fine, under X11 in DE. Here's the variables printed:

drm_fd->pixel_format: 875713112
drm_fd->width: 3840
drm_fd->height: 2160
drm_fd->fd: 14
drm_fd->offset: 0
drm_fd->pitch: 15360
drm_fd->modifier: 0

texture_rotation: 0.000000

self->base.color_conversion.params.color_range: 0
self->base.color_conversion.params.source_color: 0
self->base.color_conversion.params.destination_color: 0
self->base.color_conversion.params.load_external_image_shader: 0
self->base.color_conversion.params.num_destination_textures: 2

self->capture_size.x: 3840
self->capture_size.y: 2160
capture_pos.x: 0
capture_pos.y: 0

[dec05eba]

Hmm I dont know then, it outputs the same data as the ffmpeg terminal output. Can you try this? patch.txt also copy the terminal output of running gpu screen recorder with this change. Also just to verify, what does the gpu screen recorder command that you run look like? exactly