russelltg / wl-screenrec

High performance wlroots screen recording, featuring hardware encoding
Apache License 2.0
266 stars 9 forks source link

The selected region seems to have a bit of an issue. #26

Closed xiaohuirong closed 7 months ago

xiaohuirong commented 10 months ago

I use wf-info to obtain the region information of the window, and then pass it to wl-screenrec. I've noticed that there seems to be an unexpected border on the left side. However, when I use wf-recorder to record the same region, I don't encounter this issue. 2023-09-08_23:22:48

I can't guarantee whether wf-info has obtained the correct region, so I generated a pure black image with a resolution of 2560x1440, which matches my screen. I then added a white line at the pixel positions x=99, x=198, y=99, and y=198 on the image. black_wallpaper_with_white_line

Then, I recorded the (99,99 100x100) region separately using wl-screenrec and wf-recorder. In theory, there should have been a white pixel border in the recording. wf-recorder indeed captured it, while wl-screenrec seemed to have a bit of an issue, as shown in the following image. 2023-09-09_00:07:06

I then attempted to use wl-screenrec to record the (100,100 100x100) region to see if it was a simple offset issue. The result is as follows, the image dimensions are indeed 100x100 but it appears that it's not just a matter of offset. 2023-09-09_00:09:42

russelltg commented 7 months ago

What is your scaling setting?

xiaohuirong commented 7 months ago

My scaling is just 1.

xiaohuirong commented 7 months ago

When scaling is set to 1, it seems that the issue only occurs when the starting point of the region is an odd number. For example, with 99,100 100x100, the recording area is shifted to the left; with 100,99 100x100, the recording area is shifted upwards; with 99,99 100x100 , the recording area is shifted to the upper left. However, with 100,100 100x100 , the recorded area matches the one obtained with grim -g "100,100 100x100" and does not exhibit the aforementioned issue.

xiaohuirong commented 7 months ago

When the scaling is not 1, the same issue persists, but the patterns mentioned above may vary.

xiaohuirong commented 7 months ago

Simple script for generating test background images

#!/bin/python

from PIL import Image, ImageDraw

width, height = 2560, 1440
image = Image.new("RGB", (width, height), "black")

draw = ImageDraw.Draw(image)

line_color = (255, 255, 255)
line_y1, line_y2 = 100, 199
draw.line([(0, line_y1), (width, line_y1)], fill=line_color)
draw.line([(0, line_y2), (width, line_y2)], fill=line_color)

line_x1, line_x2 = 100, 199
draw.line([(line_x1, 0), (line_x1, height)], fill=line_color)
draw.line([(line_x2, 0), (line_x2, height)], fill=line_color)

image.save("test_image.png")
russelltg commented 7 months ago

Thanks so much for all the detailed instructions with this bug report. Unfortunately, it seems this is either a ffmpeg or vaapi bug. What hardware are you on (if you're on different hardware, then it's almost certainally a ffmpeg bug).

https://github.com/russelltg/wl-screenrec/assets/7105082/159585b0-2c75-442a-9ceb-18596eb4630d

Given this full-sceen capture (I've modified {x,y}2 to be 299, as some codecs don't like surfaces smaller than 180),

ffmpeg -vaapi_device /dev/dri/renderD129 -i screenrecord.mp4 -vf 'hwupload, crop=200:200:99:99,scale_vaapi=format=nv12:w=200:h=200' -c:v h264_vaapi croped.mp4

gives (correctly): image

ffmpeg -vaapi_device /dev/dri/renderD129 -i screenrecord.mp4 -vf 'hwupload, crop=200:200:100:100,scale_vaapi=format=nv12:w=200:h=200' -c:v h264_vaapi croped.mp4

gives (correctly):

image

ffmpeg -vaapi_device /dev/dri/renderD129 -i screenrecord.mp4 -vf 'hwupload, crop=200:200:101:101,scale_vaapi=format=nv12:w=200:h=200' -c:v h264_vaapi croped.mp4

gives (incorrectly): image

ffmpeg -vaapi_device /dev/dri/renderD129 -i screenrecord.mp4 -vf 'hwupload, crop=200:200:102:102,scale_vaapi=format=nv12:w=200:h=200' -c:v h264_vaapi croped.mp4

gives (incorrectly): image

russelltg commented 7 months ago

Actually, there's an exact flag that seems to address the situation. Do you think you could test if this patch works for you?


diff --git a/src/main.rs b/src/main.rs
index c9f11f7..b0ba2fb 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -1633,7 +1633,7 @@ fn video_filter(
         .input("out", 0)
         .unwrap()
         .parse(&format!(
-            "crop={roi_w}:{roi_h}:{roi_x}:{roi_y},scale_vaapi=format={output_real_pixfmt_name}:w={enc_w}:h={enc_h}{}",
+            "crop={roi_w}:{roi_h}:{roi_x}:{roi_y}:exact=1,scale_vaapi=format={output_real_pixfmt_name}:w={enc_w}:h={enc_h}{}",
             if let EncodePixelFormat::Vaapi(_) = pix_fmt {
                 ""
             } else {
``
russelltg commented 7 months ago

the exact=1 I think does work, however I still think there is a ffmpeg bug going on. What exact does is doesn't round to the chroma subsampling, but there is no chroma subsampling on the input image (RGB format). This snippet from ffmpeg seems to be the culprit....

    if (pix_desc->flags & AV_PIX_FMT_FLAG_HWACCEL) {
        s->hsub = 1;
        s->vsub = 1;
    } else {
        s->hsub = pix_desc->log2_chroma_w;
        s->vsub = pix_desc->log2_chroma_h;
    }

it just assumes all HW formats are 2x2 subsampled....

russelltg commented 7 months ago

Reported upstream to https://trac.ffmpeg.org/ticket/10669

xiaohuirong commented 7 months ago

This patch indeed works for me. Thank you for addressing the issue I encountered. I hope ffmpeg can fix this bug.

russelltg commented 7 months ago

Thanks for confirming. It's harmless to add (it should make no difference in this situation, but does due to a bug), so I just pushed e0945ebed09be1a819217f7316ddae30fb82989e. Again, thanks so much for such a detailed report! Sorry it took me a bit to get to.