search

redrathnure / armbian-mkspi

Armbian Linux Build Framework
https://www.armbian.com
GNU General Public License v2.0
83 stars 13 forks source link

Does anyone have hardware video encoding working? #30

Closed alexvnesta closed 2 weeks ago

alexvnesta commented 1 month ago

Which feature would you like to have?

I have an MKS SKIPR. I've messed with the software a bunch trying to get it functional. Currently I have everything up to date and working properly except for hardware video encoding for the webcam.

I see linux kernels 4.4 and 5.10 have official video encoding support via adding MPP into the kernel. I also see that the now release candidate linux kernel 6.10 has improved support for newer rockchip mali series video chips.

I'm still plugging away at this, but if someone has it figured out please let me know!

Currently I am on armbian bookworm with kernel 6.7.

Thanks!

Funding

redrathnure commented 1 month ago

Good question... Kind of related questions:

alexvnesta commented 1 month ago

@redrathnure

  1. Official armbian has hardware video acceleration for a very dated debian jammy build as far as I know.

  2. official MKS builds do not have hardware video acceleration for the builds based on kernel 5.16. There is a build with kernel 4.4 that might have acceleration, but I do not want to re-flash my SKIPR board to test it.

  3. It would be nice to try the latest 6.10 RC kernel, but I am not 100% certain everything will "just work". The hardware with confirmed compatibility were the newer RK3399 boards. I think this is all mainly achieved through integration of the "panfrost open source mali video driver".

Here is a note about adding panfrost just about a month ago. https://lore.kernel.org/all/20240531-mesa-panfrost-v1-0-84e23ae9a600@cherry.de/T/

You can see rockchip testing notes on the more recent 6.10 kernel pull requests. Here is one. https://patchwork.kernel.org/project/linux-rockchip/patch/10237789.nnTZe4vzsl@diego/

There are others as well. I am not an expert this is just what I have found digging through the internet.

I have an RK3328 libre renegade board available for testing - I know there are some small differences with the mks skipr, but the hardware video encoding portion should be the same.

I can't find a pre-compiled 6.10 kernel version of armbian to test on my renegade board. I imagine in just a few weeks there may be one. (edge updates were kernel 6.9, and rockchip mali gpu support was added in 6.10 rc5).

redrathnure commented 1 month ago

How to test it? Is it possible verify HW acceleration from CLI without X/Wayland installed?

alexvnesta commented 1 month ago

Ok, I think I learned a little more about the topic. What we need to do webcam hardware encoding is ffmpeg with rockchip support. ffmpeg requires rockchip mpp. Rockchip mpp requires the rockchip fork of kernel 5.10.

Recently, kernel 6.1 BSP was released, and this can be used to generate an armbian 6.1 (NOT 6.10-RC) kernel with mpp support.

see here https://forum.radxa.com/t/bsp-kernel-6-1-releasing-today/19877 and here: https://github.com/radxa/kernel/tree/linux-6.1-stan-rkr1 and details on ffmpeg with mmp: https://forum.radxa.com/t/updated-ffmpeg-with-mpp/17771/54

If you can get armbian with the 6.1 kernel BSP compiled, then you can test mpp and ffmpeg with this:

ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 -c:v mjpeg_rkmpp -qscale 10 -f mjpeg output_hw.mjpeg
alexvnesta commented 1 month ago

I have made a TON of progress this week.

1) we need MPP - this is totally separate from the mali gpu. 2) Recently a 6.1 board support package (BSP) was released for rockchip devices for the linux kernel (v6.1 only). This enables the use of MPP on kernel 6.1! 3) This kernel was adapted to work on armbian - see https://github.com/armbian/linux-rockchip - the rk-6.1-rkr1 branch - rkr3 coming soon. 4) I am in the process of adapting the mkspi patches @redrathnure generated/refined for this kernel 6.1. I also spent days configuring the kernel for the rk3328.

The image builds, but there are still some weird issues with zram and zswap - so the first boot is painfully long. There are probably other modules that need to be sorted. If someone really wants the changes I made, let me know. I can't guaruntee they are correct - I am figuring this out as I go and referring to the rk3328 datasheet.

Once I get MPP + encoding working I will post an update :)

Unfortunately, since this is just kernel 6.1 and not mainline, it may be out of scope of this repo. Please let me know.

alexvnesta commented 2 weeks ago

After weeks of effort, I gave up on this for now. It looks like there are just too many layers of incompatibility. I think rockchip is slowly incorporating some of the necessary changes into the armbian kernel 6.1 BSP and ffmpeg, but I don't want to spend much more effort/time on this.

Here is a set of recently submitted ffmpeg patches: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=12604

And here is a thread to follow for someone else trying to get hardware encoding on an rk3328 based board. https://github.com/blakeblackshear/frigate/discussions/12577