Pink tint with drmprime/dmabuf on Lima driver (X11/Weston/tty)

paolosabatino commented 9 months ago

Important Information

Provide following Information:

mpv version: 0.35.1
Linux Distribution and Version: debian bookworm, ubuntu jammy
Source of the mpv binary: distribution source code
If known which version of mpv introduced the problem: probably 0.35
Window Manager and version: xfce, Weston, bare terminal
GPU driver and version: Mesa git (23.2) via oibaf PPA on Ubuntu, Mesa 22.3 on Debian Bookworm
Possible screenshot or video of visual glitches:

mpv is version 0.35.1 because it is the packaged binary that comes with Debian Bookworm.

My intent was to provide an apt repository with ffmpeg compiled with v4l2-request and v4l2-drmprime patches from LibreElec project to ease the fruition and testing of such features on ready-made distribution images (for instace debian/ubuntu images built by Armbian project) for ARM devices. The repository is still experimental and under testing, but has already proved to work for arm64 and amrhf architectures (see below for setup instructions)

I'm not exactly sure the issue presented here is related to mpv or mesa driver, but I would like to discover and possibly excalate.

Reproduction steps

Preconditions:

a running system with Debian Bookworm or Ubuntu Jammy, or any other suitable distro
Mali Utgard 400/450 hardware
ffmpeg (>= 4.4.2) with v4l2-request and v4l2-drmprime patches
mpv version 0.35.1 installed

Run an h.264/h.265 video with the following command lines:

# Hardware decoded video running in a X11 window (tested on xfce), presentation via EGL
mpv --vo=gpu --hwdec=drm --gpu-context=x11egl --gpu-hwdec-interop=drmprime <your-video.mp4>

# Hardware decoded video running in a window of a Wayland compositor (tested on weston), presentation via EGL
mpv --vo=gpu --hwdec=drm --gpu-context=wayland --gpu-hwdec-interop=drmprime  <your-video.mp4>

# Hardware decoded video running on bare terminal, presentation via EGL
mpv --vo=gpu --hwdec=drm --gpu-context=drm --gpu-hwdec-interop=drmprime  <your-video.mp4>

Additional details:

the command lines supplied above all make use of hardware decoding and dmabuf features, with smooth reproduction of the content and reasonably low cpu usage even on slower armhf SoCs.
Debian Bookworm comes with ffmpeg 5.1.4, Ubuntu Jammy with ffmpeg 4.4.2; on both of them the issue is appearing
the pink tint has been experimented on various SoCs, in particular I tested rockchip rk3229 (armhf, mali400, rkvdec hardware decoder), rk3328 (arm64, mali450, rkvdec hardware decoder) and allwinner H3 (armhf, mali400, cedrus hardware decoder) and all of them exhibit the very same issue
testing on SoCs like rk3288 (armhf, mali T760) and rk3399 (arm64, mali T860) does not trigger the same issue, presumably because the GPU is handled by panfrost and not lima driver
Using mpv in a "direct to plane" fashion in a virtual terminal with mpv --vo=gpu --hwdec=drm --gpu-context=drm --gpu-hwdec-interop=drmprime-overlay <your-video.mp4> does not show the issue
Using --hwdec=drm-copy does not show the issue, although playback is choppy depending upon the SoC power

To make testing easier, I can provide the details of my ffmpeg-v4l2request testing repository ~~but I would prefer to be contacted in private via email since it is temporarely hosted on a server in my private LAN.~~ and instructions to setup are available on Armbian forum here

Expected behavior

The video is played with correct colors

Actual behavior

The video colors are biased to pink: full white is pink, full black is red; as can be seen from the screenshot, it affects only the texture of the video and not the HUD overlay.

Log file

mpv.log

Sample files

Any h.264 or h.265 video file that can trigger the hardware decoding capabilities of the SoC.

philipl commented 9 months ago

To the best of our knowledge, this is a Lima bug. The same mpv code produces correct results on panfrost and rpi.

WeihuaGu commented 9 months ago

the same with:inxi -G Graphics: Device-1: display-subsystem driver: rockchip_drm v: N/A Device-2: rk3328-mali driver: lima v: kernel Device-3: rk3328-dw-hdmi driver: dwhdmi_rockchip v: N/A Display: wayland server: X.org v: 1.21.1.9 with: Xwayland v: 23.2.2 compositor: sway v: 1.8.1 driver: N/A resolution: 1280x720~50Hz API: EGL v: 1.4,1.5 drivers: lima,rockchip,swrast platforms: gbm,wayland,x11,surfaceless,device API: OpenGL v: 2.1 vendor: mesa v: 23.2.1-1 renderer: Mali450

and ffmpeg build from git@github.com:Kwiboo/FFmpeg.git with version n6.1-15-gba52791ba7;mpv version v0.37.0-25-gdea512ea38

bigrob3rto commented 8 months ago

Found that forcing MESA to a different GL core profile can solve the pinkish problem:

export MESA_GL_VERSION_OVERRIDE=3.2COMPAT export MESA_GLSL_VERSION_OVERRIDE=320

mpv your-video.mkv

paolosabatino commented 8 months ago

It seems that just forcing the OpenGL 3.0+ code path in place of OpenGL 2.1 fixes the issue, at the expense of some opengl error messages during runtime.

This command line works enough for me (full logfile here): MESA_GL_VERSION_OVERRIDE=3.0 mpv --hwdec=drm your-video.mkv

But also setting fbo-format to rgba16 (or rgba8) throws away some fbo-format autodetection errors (rgba16f seems not to be supported at all): MESA_GL_VERSION_OVERRIDE=3.0 mpv --hwdec=drm --fbo-format=rgba16 your-video.mkv

paolosabatino commented 8 months ago

Another working code path is using OpenGL ES 3.0+, which produces an even cleaner output: MESA_GLES_VERSION_OVERRIDE=3.0 mpv --hwdec=drm --opengl-es=yes --fbo-format=rgba8 your-video.mkv

zhangn1985 commented 8 months ago

use mpv -vo=gpu --gpu-hwdec-interop=drmprime-overlay to play

I tested on H3, with mpv 0.37.0, this no need GL override.

philipl commented 8 months ago

The overlay mechanism bypasses OpenGL so it isn't affected by whatever Lima bug is in play here

paolosabatino commented 8 months ago

@zhangn1985 true, but drmprime-overlay does not use EGL/OpenGL to present the frames, so Lima is not involved at all, everything is done at DRM level. Correct drmprime-overlay functioning is already cited in the issue description.

na-na-hi commented 6 months ago

The same happens on Intel iGPU, Mesa 22.3.6 (Debian 12): GLES 2.0 and vaapi hwdec results in the same pink/red tint. Tested on Haswell.

MESA_GLES_VERSION_OVERRIDE=2.0 mpv --hwdec=vaapi --opengl-es=yes video.mkv

llyyr commented 6 months ago

Can also reproduce on radeonsi if I intentionally force gles version to 2.0 through the Mesa variable

anarsoul commented 1 month ago

It's mpv bug, and rather trivial one. in fragment shader you have:

color.gb = 1.000000 * vec4(texture(texture1, texcoord1).ra;

While it should be:

color.gb = 1.000000 * vec4(texture(texture1, texcoord1).rg;

Since external format is RG88 according to fourcc passed to eglCreateImageKHR().

If I fix the shader in apitrace capture, it replays correctly. Apparently for GLES2 case, MPV tries to sample RG texture as if it was luminance alpha texture.

anarsoul commented 1 month ago

Apparently it extracts data into RGBA texture via EGLImageTargetTexture2DOES, but then it thinks it's an LA texture when it generates fragment shader. I guess I'll let mpv devs to debug it further.

anarsoul commented 1 month ago

Possible workaround is to use GL_R8 and GL_RG8 formats if these are available, since I doubt that anyone is testing L8 and LA8 codepath.

Modern drivers for GL2-class hardware (lima, vc4) support more formats by doing swizzling in the shader, so there is no good reason to use L and LA8. It might be worth to drop L/LA8 format supports completely.

See PR above.

anarsoul commented 1 month ago

See PR above.

It won't work on Lima, since it doesn't have ARB_framebuffer_object (since HW doesn't support attachments with different dimensions).

mpv may get away with EXT_framebuffer_object instead, but it would require more invasive changes.

mpv-player / mpv