Closed ctrlcctrlv closed 10 months ago
It seems that iHD and mesa is reporting exactly opposite values for blue_mask, green_mask and red_mask. For iHD it is always MSB_FIRST while mesa reports LSB_FIRST. The plan here is to always use the requested format and blindly assume the vaapi driver returns the correct image format.
Also can you please provide the output of kmsvnc --debug -i
[fred@大.狸.agency ~]$ sudo !!
sudo kmsvnc --debug -i
drm driver is amdgpu
Plane 45 CRTC 0 FB 0 Type primary
Plane 50 CRTC 0 FB 0 Type primary
Plane 55 CRTC 0 FB 0 Type primary
Plane 60 CRTC 0 FB 0 Type primary
Plane 65 CRTC 90 FB 153 Type primary
Using plane 65 to locate framebuffers
Template framebuffer is 153: 4864x2160 fourcc:875713112 mod:144115188622392067 flags:2
handles 1 1 1 0
offsets 0 42336256 42541056 0
pitches 19456 5120 5120 0
format XR24, modifier AMD:GFX10_RBPLUS,GFX9_64K_R_X,DCC,DCC_RETILE,DCC_INDEPENDENT_64B,DCC_INDEPENDENT_128B,DCC_MAX_COMPRESSED_BLOCK=64B,DCC_CONSTANT_ENCODE,PIPE_XOR_BITS=4,PACKERS=4
va msg: VA-API version 1.20.0
va msg: Trying to open /usr/lib/dri/radeonsi_drv_video.so
va msg: Found init function __vaDriverInit_1_20
va msg: va_openDriver() returns 0
vaapi vendor Mesa Gallium driver 23.3.0-devel for AMD Radeon RX 6900 XT (navi21, LLVM 16.0.6, DRM 3.52, 6.0.0-1-hdr-git-02753-gff57328e5784)
selected rt_format 131072, alpha 0
image fmt: fourcc 842094158, NV12, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 808530000, P010, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 909193296, P016, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 808596553, I420, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 842094169, YV12, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1448695129, YUYV, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 844715353, YUY2, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1498831189, UYVY, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 808466521, Y800, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1345598516, 444P, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1346520914, RGBP, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1095911234, BGRA, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, reserved 0 0 0 0
image fmt: fourcc 1094862674, RGBA, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, reserved 0 0 0 0
image fmt: fourcc 1111970369, ARGB, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, reserved 0 0 0 0
image fmt: fourcc 1481787202, BGRX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, reserved 0 0 0 0
image fmt: fourcc 1480738642, RGBX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, reserved 0 0 0 0
va operation error 0x1 operation failed on line 338
got vaapi image:
image fmt: fourcc 1481787202, BGRX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, reserved 0 0 0 0
18/09/2023 08:36:34 Listening for VNC connections on TCP port 5900
18/09/2023 08:36:34 Listening for VNC connections on TCP6 port 5900
^C[fred@大.狸.agency ~]$
It seems that mesa returns nonsense blue_mask, green_mask and red_mask.
va msg: VA-API version 1.20.0
va msg: User environment variable requested driver 'iHD'
va msg: Trying to open /usr/lib/dri/iHD_drv_video.so
va msg: Found init function __vaDriverInit_1_20
va msg: va_openDriver() returns 0
vaapi vendor Intel iHD driver for Intel(R) Gen Graphics - 23.3.3 ()
selected rt_format 131072, alpha 0
image fmt: fourcc 1095911234, BGRA, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff000000, green_mask 0xff0000, red_mask 0xff00, reserved 0 0 0 0
image fmt: fourcc 1111970369, ARGB, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, reserved 0 0 0 0
image fmt: fourcc 1094862674, RGBA, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff00, green_mask 0xff0000, red_mask 0xff000000, reserved 0 0 0 0
image fmt: fourcc 1380401729, ABGR, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, reserved 0 0 0 0
image fmt: fourcc 1481787202, BGRX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff000000, green_mask 0xff0000, red_mask 0xff00, reserved 0 0 0 0
image fmt: fourcc 1111970392, XRGB, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, reserved 0 0 0 0
image fmt: fourcc 1480738642, RGBX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff00, green_mask 0xff0000, red_mask 0xff000000, reserved 0 0 0 0
image fmt: fourcc 1380401752, XBGR, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, reserved 0 0 0 0
image fmt: fourcc 808669761, AR30, byte_order VA_LSB_FIRST, bpp 32, depth 30, blue_mask 0x3ff, green_mask 0xffc00, red_mask 0x3ff00000, reserved 0 0 0 0
image fmt: fourcc 808665665, AB30, byte_order VA_LSB_FIRST, bpp 32, depth 30, blue_mask 0x3ff00000, green_mask 0xffc00, red_mask 0x3ff, reserved 0 0 0 0
image fmt: fourcc 808669784, XR30, byte_order VA_LSB_FIRST, bpp 32, depth 30, blue_mask 0x3ff, green_mask 0xffc00, red_mask 0x3ff00000, reserved 0 0 0 0
image fmt: fourcc 808665688, XB30, byte_order VA_LSB_FIRST, bpp 32, depth 30, blue_mask 0x3ff00000, green_mask 0xffc00, red_mask 0x3ff, reserved 0 0 0 0
image fmt: fourcc 909199186, RG16, byte_order VA_LSB_FIRST, bpp 16, depth 16, blue_mask 0x1f, green_mask 0x7e0, red_mask 0xf800, reserved 0 0 0 0
image fmt: fourcc 1346520914, RGBP, byte_order VA_LSB_FIRST, bpp 24, depth 24, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1347569474, BGRP, byte_order VA_LSB_FIRST, bpp 24, depth 24, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1448433985, AYUV, byte_order VA_LSB_FIRST, bpp 32, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 808466521, Y800, byte_order VA_LSB_FIRST, bpp 8, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 842094158, NV12, byte_order VA_LSB_FIRST, bpp 12, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 825382478, NV21, byte_order VA_LSB_FIRST, bpp 12, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 844715353, YUY2, byte_order VA_LSB_FIRST, bpp 16, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1498831189, UYVY, byte_order VA_LSB_FIRST, bpp 16, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 842094169, YV12, byte_order VA_LSB_FIRST, bpp 12, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 808596553, I420, byte_order VA_LSB_FIRST, bpp 12, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1345401140, 411P, byte_order VA_LSB_FIRST, bpp 12, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1211249204, 422H, byte_order VA_LSB_FIRST, bpp 16, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1446130228, 422V, byte_order VA_LSB_FIRST, bpp 16, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 1345598516, 444P, byte_order VA_LSB_FIRST, bpp 24, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 860048713, IMC3, byte_order VA_LSB_FIRST, bpp 16, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
image fmt: fourcc 808530000, P010, byte_order VA_LSB_FIRST, bpp 24, depth 0, blue_mask 0, green_mask 0, red_mask 0, reserved 0 0 0 0
got vaapi image:
image fmt: fourcc 1480738642, RGBX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff00, green_mask 0xff0000, red_mask 0xff000000, reserved 0 0 0 0
18/09/2023 21:09:54 Listening for VNC connections on TCP port 5900
18/09/2023 21:09:54 Listening for VNC connections on TCP6 port 5900
Looks like I'll have to hard code these masks myself.
Should be fixed by 4ba2de0
sorry still broken :s
[fred@大.狸.agency ~/.cache/yay/kmsvnc-git]$ sudo kmsvnc --debug -i
drm driver is amdgpu
Plane 40 CRTC 0 FB 0 Type primary
Plane 45 CRTC 0 FB 0 Type primary
Plane 50 CRTC 0 FB 0 Type primary
Plane 55 CRTC 0 FB 0 Type primary
Plane 60 CRTC 0 FB 0 Type primary
Plane 65 CRTC 77 FB 119 Type primary
Using plane 65 to locate framebuffers
Template framebuffer is 119: 3840x2160 fourcc:875713112 mod:0 flags:2
handles 1 0 0 0
offsets 0 0 0 0
pitches 15360 0 0 0
format XR24, modifier NONE:LINEAR
va msg: VA-API version 1.20.0
va msg: Trying to open /usr/lib/dri/radeonsi_drv_video.so
va msg: Found init function __vaDriverInit_1_20
va msg: va_openDriver() returns 0
vaapi vendor Mesa Gallium driver 23.3.0-devel for AMD Radeon RX 6900 XT (navi21, LLVM 16.0.6, DRM 3.49, 6.1.54-1-lts)
selected rt_format 131072, alpha 0
image fmt: fourcc 842094158, NV12, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 808530000, P010, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 909193296, P016, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 808596553, I420, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 842094169, YV12, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 1448695129, YUYV, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 844715353, YUY2, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 1498831189, UYVY, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 808466521, Y800, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 1345598516, 444P, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 1346520914, RGBP, byte_order VA_MSB_FIRST, bpp 0, depth 0, blue_mask 0, green_mask 0, red_mask 0, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 1095911234, BGRA, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, alpha_mask 0xff000000, reserved 0 0 0 0
image fmt: fourcc 1094862674, RGBA, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, alpha_mask 0xff000000, reserved 0 0 0 0
image fmt: fourcc 1111970369, ARGB, byte_order VA_LSB_FIRST, bpp 32, depth 32, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, alpha_mask 0xff000000, reserved 0 0 0 0
image fmt: fourcc 1481787202, BGRX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, alpha_mask 0, reserved 0 0 0 0
image fmt: fourcc 1480738642, RGBX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff0000, green_mask 0xff00, red_mask 0xff, alpha_mask 0, reserved 0 0 0 0
va operation error 0x1 operation failed on line 386
got vaapi image:
image fmt: fourcc 1481787202, BGRX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff, green_mask 0xff00, red_mask 0xff0000, alpha_mask 0, reserved 0 0 0 0
selected image format:
image fmt: fourcc 1481787202, BGRX, byte_order VA_LSB_FIRST, bpp 32, depth 24, blue_mask 0xff000000, green_mask 0xff0000, red_mask 0xff00, alpha_mask 0, reserved 0 0 0 0
24/09/2023 07:47:46 Listening for VNC connections on TCP port 5900
24/09/2023 07:47:46 Listening for VNC connections on TCP6 port 5900
24/09/2023 07:47:53 0 other clients
24/09/2023 07:47:53 Normal socket connection
24/09/2023 07:47:53 Client Protocol Version 3.8
24/09/2023 07:47:53 Protocol version sent 3.8, using 3.8
24/09/2023 07:47:53 rfbProcessClientSecurityType: executing handler for type 1
24/09/2023 07:47:53 rfbProcessClientSecurityType: returning securityResult for client rfb version >= 3.8
24/09/2023 07:47:53 Pixel format for client ::1:
24/09/2023 07:47:53 32 bpp, depth 24, little endian
24/09/2023 07:47:53 true colour: max r 255 g 255 b 255, shift r 16 g 8 b 0
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEC6)
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0x574D5664)
24/09/2023 07:47:53 Enabling full-color cursor updates for client ::1
24/09/2023 07:47:53 Enabling X-style cursor updates for client ::1
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0x574D5666)
24/09/2023 07:47:53 Enabling NewFBSize protocol extension for client ::1
24/09/2023 07:47:53 Enabling ExtDesktopSize protocol extension for client ::1
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEFB)
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0x574D5668)
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFECD)
24/09/2023 07:47:53 Enabling LastRect protocol extension for client ::1
24/09/2023 07:47:53 Enabling ExtendedClipboard extension for client ::1
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEC7)
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEC8)
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEFE)
24/09/2023 07:47:53 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0x00000032)
24/09/2023 07:47:53 Using compression level 2 for client ::1
24/09/2023 07:47:53 Using image quality level 8 for client ::1
24/09/2023 07:47:53 Using JPEG subsampling 0, Q92 for client ::1
24/09/2023 07:47:53 Using tight encoding for client ::1
24/09/2023 07:47:53 Sending rfbEncodingExtDesktopSize for size (3840x2160)
24/09/2023 07:47:53 Client requested resolution change to (1896x1034)
24/09/2023 07:47:53 Sending rfbEncodingExtDesktopSize for size (3840x2160) resize prohibited
24/09/2023 07:47:53 Client requested resolution change to (1888x999)
24/09/2023 07:47:54 Sending rfbEncodingExtDesktopSize for size (3840x2160) resize prohibited
24/09/2023 07:47:56 Client ::1 gone
24/09/2023 07:47:56 Statistics events Transmit/ RawEquiv ( saved)
24/09/2023 07:47:56 ServerCutText : 1 | 16/ 16 ( 0.0%)
24/09/2023 07:47:56 FramebufferUpdate : 16 | 0/ 0 ( 0.0%)
24/09/2023 07:47:56 LastRect : 1 | 12/ 12 ( 0.0%)
24/09/2023 07:47:56 tight : 128 | 34142/ 33181056 ( 99.9%)
24/09/2023 07:47:56 RichCursor : 1 | 255/ 255 ( 0.0%)
24/09/2023 07:47:56 ExtendedDesktopSize : 3 | 96/ 96 ( 0.0%)
24/09/2023 07:47:56 TOTALS : 150 | 34521/ 33181435 ( 99.9%)
24/09/2023 07:47:56 Statistics events Received/ RawEquiv ( saved)
24/09/2023 07:47:56 PointerEvent : 28 | 168/ 168 ( 0.0%)
24/09/2023 07:47:56 SetDesktopSize : 2 | 48/ 48 ( 0.0%)
24/09/2023 07:47:56 ClientCutText : 1 | 16/ 16 ( 0.0%)
24/09/2023 07:47:56 FramebufferUpdate : 20 | 200/ 200 ( 0.0%)
24/09/2023 07:47:56 SetEncodings : 1 | 104/ 104 ( 0.0%)
24/09/2023 07:47:56 SetPixelFormat : 1 | 20/ 20 ( 0.0%)
24/09/2023 07:47:56 TOTALS : 53 | 556/ 556 ( 0.0%)
^C[fred@大.狸.agency ~/.cache/yay/kmsvnc-git]$