search

JeffyCN / libv4l-rkmpp

A rockchip-mpp V4L2 wrapper plugin for chromium V4L2 VDA/VEA
GNU Lesser General Public License v2.1
77 stars 25 forks source link

Handling 10-bit HEVC/VP9/AV1 videos? #21

Open nyanmisaka opened 2 months ago

nyanmisaka commented 2 months ago

Some self-hosted media servers and Youtube/Bilibili can query 10-bit video decode support via HTMLMediaElement: canPlayType() reported by the browser and play 10-bit/HDR directly in Chromium.

If I understand correctly, the current libv4l-rkmpp code base only handles 8-bit video. For 10-bit 4:2:0 videos, the MPP decoded NV15 memory will be displayed as NV12 (corrupted image for sure) since NV15 is a compact version of P010, which is not supported by EGL and Mesa.

IMO the NV15 can be converted to P010 with RGA3 before importing into EGL to avoid loss of precision resulting in color banding, or fallback to using NV12 to ensure compatibility.

So, are there any plans to improve support for 10-bit video (HEVC Main10, VP9 Profile 2 and AV1 Main)? Thanks in advance.

image

JeffyCN commented 2 months ago

there's no plane to use rga now, it has too many limits and changes all the time, so i would rather try the gpu(mali ddk supports those formats).

nyanmisaka commented 2 months ago

there's no plane to use rga now, it has too many limits and changes all the time, so i would rather try the gpu(mali ddk supports those formats).

Either way is fine for me. Could you add support for V4L2_PIX_FMT_NV15 and make use of it in libmali EGL (mali ddk)? Mesa (collabora panthor) may also provide support for NV15 in the future, but that is another story.

JeffyCN commented 2 months ago

i'll try to do it next week, maybe try to add rga(using legacy api) as well.

JeffyCN commented 2 months ago

btw, could you provide some sample videos for verifying?

nyanmisaka commented 2 months ago

https://kodi.wiki/view/Samples#4K_(UltraHD)_Formats You can grab some from here, which requires you to have access to google drive or mega.nz. Or use some random promos made for TV, which are usually 10-bit (with or without HDR).

I don't know how rockchip handles HDR on Linux. So playing it without tonemapping may result in pale colors. AFAIK Chromium is able to handle this via GPU on some platforms. There is no such concern for 10-bit SDR videos.