4GB limit for hardware video processing

[MarcA711]

What happened?

The hardware for video processing of the rk3588 (and possibly other socs as well) is only able to address 4GB at the beginning of the memory (see https://github.com/rockchip-linux/mpp/issues/560). This means that video processing will crash if this memory range is allocated by other processes. dmesg will show a long error, this is an excerpt:

[350493.782104] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=docker-0e6cc68fa5b937fbd8ec748366f4e74e775694ceb154b60df0eff524cc1e1960.scope,mems_allowed=0,global_oom,task_memcg=/system.slice/docker-0e6cc68fa5b937fbd8ec748366f4e74e775694ceb154b60df0eff524cc1e1960.scope,task=frigate.capture,pid=2834,uid=0
[350493.782138] Out of memory: Killed process 2834 (frigate.capture) total-vm:18243892kB, anon-rss:11574836kB, file-rss:2040kB, shmem-rss:1360kB, UID:0 pgtables:23140kB oom_score_adj:0
[350493.784593] mali fb000000.gpu: OOM notifier: dev mali0  0 kB

This is very annoying and makes hardware video processing unusable for me.

Rockchip published a patch that should fix the issue: https://github.com/rockchip-linux/mpp/issues/560#issuecomment-2164189090

However, I haven't tested this patch. I have no idea how to compile the kernel and install it on my board. Are there instructions for this? Maybe someone can help testing it.

If this patch actually works, it should be applied to the kernel.

Kernel version

6.1.0-1020-rockchip

SBC model

Orange Pi 5 Plus (16GB)

What operating system are you seeing this problem on?

Ubuntu 22.04 LTS (Jammy Jellyfish), Ubuntu 24.04 LTS (Noble Nombat)

Relevant logs

No response

[nyanmisaka]

git clone --depth=1 https://github.com/Joshua-Riek/linux-rockchip
cd linux-rockchip
git apply *.patches
mk-build-deps -i
dpkg-buildpackage -b -us -uc -rfakeroot

This can take a long time on a full compile. Then you will get a deb package containing the kernel image.

[MarcA711]

I don't know if its my fault, but I had no success to built the package:

# mk-build-deps -i
Usage:
    mk-build-deps --help|--version

    mk-build-deps [options] control file | package name ...

I tried mk-build-deps -i linux-rockchip instead and it worked.

However, the next command fails and I have no clue how to fix it:

$ dpkg-buildpackage -b -us -uc -rfakeroot
dpkg-buildpackage: error: cannot open file debian/changelog: No such file or directory

[great9]

for those of us who installed focal (ubuntu 20.04 LTS) thinking radxa will support their board we have even more problems as linux-rockchip src package doesn't even exist (and deb-src is unavailable in general)

[great9]

However, the next command fails and I have no clue how to fix it:

$ dpkg-buildpackage -b -us -uc -rfakeroot
dpkg-buildpackage: error: cannot open file debian/changelog: No such file or directory

you can just create an empty file

touch debian/changelog

or add one line into it

echo "testing DMABUF kernel fixes for rk3588" > debian/changelog

[Joshua-Riek]

for those of us who installed focal (ubuntu 20.04 LTS) thinking radxa will support their board we have even more problems as linux-rockchip src package doesn't even exist (and deb-src is unavailable in general)

I hope you are aware Radxa does not support Ubuntu. It's only built to confirm their CI works..

[great9]

for those of us who installed focal (ubuntu 20.04 LTS) thinking radxa will support their board we have even more problems as linux-rockchip src package doesn't even exist (and deb-src is unavailable in general)

I hope you are aware Radxa does not support Ubuntu. It's only built to confirm their CI works..

back when i got my rock5b from radxa I followed what they had in their guides and installed ubuntu focal. maybe they've removed it since then.

what are my options now except reinstalling ubuntu 24 from your repo?

[Joshua-Riek]

for those of us who installed focal (ubuntu 20.04 LTS) thinking radxa will support their board we have even more problems as linux-rockchip src package doesn't even exist (and deb-src is unavailable in general)

I hope you are aware Radxa does not support Ubuntu. It's only built to confirm their CI works..

back when i got my rock5b from radxa I followed what they had in their guides and installed ubuntu focal. maybe they've removed it since then.

what are my options now except reinstalling ubuntu 24 from your repo?

Uhhhh your options are fixing the problem yourself. Im taking a very long break from this project.

[SimplyCorbett]

for those of us who installed focal (ubuntu 20.04 LTS) thinking radxa will support their board we have even more problems as linux-rockchip src package doesn't even exist (and deb-src is unavailable in general)

I hope you are aware Radxa does not support Ubuntu. It's only built to confirm their CI works..

back when i got my rock5b from radxa I followed what they had in their guides and installed ubuntu focal. maybe they've removed it since then. what are my options now except reinstalling ubuntu 24 from your repo?

Uhhhh your options are fixing the problem yourself. Im taking a very long break from this project.

Err... I'm happy for you, but will security fixes still come through?

If not, well, that's going to suck. But I'll deal with it.

[nyanmisaka]

Unfortunately this patch does not solve the issue. The error still pops up when the total amount of allocated drm dumb buf or dma-buf memory exceeds 4GB. *ERROR* out of I/O virtual memory: -28

One can reproduce the issue with this command using ffmpeg-rockchip.

# Allocate 32 times 7680x4320 RGBA buffer, obviously it exceeds the 4GB.

./ffmpeg -init_hw_device rkmpp=hw:,dma32=0 \
-f lavfi -i nullsrc=s=7680x4320,format=rgba -vf hwupload=extra_hw_frames=32 -f null -

[great9]

Unfortunately this patch does not solve the issue. The error still pops up when the total amount of allocated drm dumb buf or dma-buf memory exceeds 4GB. *ERROR* out of I/O virtual memory: -28

It helps most of the use-cases by flashing the buffer. If somebody is going to try to run 32 x 4k like from your example off of an SBC, we're not there yet with hardware in general.

[nyanmisaka]

If somebody is going to try to run 32 x 4k like from your example off of an SBC, we're not there yet with hardware in general.

You have to know that decoding an 8k HEVC video requires 20 times 7680x4320 DPB (Decoded Picture Buffer). So this is not an exaggeration. I just changed NV12/P010 to RGBA to trigger the error faster.