Bug Report: System Freezing When Running More Than 3 RGA Threads on Orange Pi 5 (RK3588S)

[thuongton999]

What happened?

I am developing a GStreamer-based application on the Orange Pi 5 (RK3588) that runs multiple GStreamer pipelines in parallel, each on a different thread. The pipelines are designed to handle video streams from different RTSP sources, utilizing the Rockchip RGA for hardware-accelerated video processing.

When running up to 3 threads, the system functions as expected. However, when I attempt to run more than 3 threads, the system begins to log repeated RGA_BLIT failure messages and eventually freezes, requiring a hard reset. Below is the error message encountered:

RgaBlit(1485) RGA_BLIT fail: Invalid argument
RgaBlit(1486) RGA_BLIT fail: Invalid argument
handl-fd-vir-phy-hnd-format[0, 209, (nil), (nil), 0, 0]
rect[0, 0, 1920, 1080, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
handl-fd-vir-phy-hnd-format[0, 0, 0xfffef00ee9f0, (nil), 0, 0]
rect[0, 0, 640, 480, 640, 480, 512, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]

Here is my sample GStreamer pipeline:

gst-launch-1.0 rtspsrc location=rtsp://<URI> ! queue ! rtph264depay ! h264parse ! mppvideodec ! rgaconvert ! videorate ! video/x-raw,format=RGB,width=640,height=480,framerate=15/1 ! fakesink

Expected Behavior: The system should be able to handle multiple threads utilizing the RGA without freezing or logging continuous errors.

Actual Behavior: When more than 3 threads are run, the system logs repeated RGA_BLIT errors and eventually freezes.

Environment:

• Board: Orange Pi 5 (RK3588S)
• OS: Ubuntu Rockchip 22.04

Kernel version

5.10.0-1012-rockchip

SBC model

Orange Pi 5

What operating system are you seeing this problem on?

Ubuntu 22.04 LTS (Jammy Jellyfish)

Relevant logs

No response

[LEET00]

I have the same problem using rock5B. When launching multiple streams the application freezes. I can still access the system when it happens. I tested with ubuntu 22.04 and ubuntu 24.04. I can see some RGA errors when it happens. dmesg.log journalctl.log

[nyanmisaka]

The RGA2 hardware on RK3588 only supports access to address space within 4GB, and the scheduler may assign tasks to RGA3 or RGA2. At this time, RGA2 will fail.

The solution is to set the allocation flag of the RGA input/output buffer to DMA32 (that is, the decoder buffer and an internal buffer used by the encoder), or set the RGA core to only RGA3 0/1 core.

• Add | MPP_BUFFER_FLAGS_DMA32 to https://github.com/JeffyCN/rockchip_mirrors/blob/c37e7cf10283521c262f9e71fd9be0422a457989/gst/rockchipmpp/gstmppallocator.c#L288

• Add dst_info.core = 0x3; // (IM_SCHEDULER_RGA3_CORE0 | IM_SCHEDULER_RGA3_CORE1) to https://github.com/JeffyCN/rockchip_mirrors/blob/c37e7cf10283521c262f9e71fd9be0422a457989/gst/rockchipmpp/gstmpp.c#L309

[LEET00]

I did the modifications and rebuild the plugins. Now when the stream starts i see the errors : [ 268.974155] mpp_rkvdec2 fdc48100.rkvdec-core: resetting for err 0x23 [ 268.974336] mpp_rkvdec2 fdc48100.rkvdec-core: reset done [ 268.974346] mpp_rkvdec2 fdc38100.rkvdec-core: resetting for err 0x107 [ 268.974524] mpp_rkvdec2 fdc38100.rkvdec-core: reset done [ 271.587089] mpp_rkvdec2 fdc48100.rkvdec-core: resetting for err 0x107 [ 271.587252] mpp_rkvdec2 fdc48100.rkvdec-core: reset done [ 271.587259] mpp_rkvdec2 fdc38100.rkvdec-core: resetting for err 0x23 [ 271.587389] mpp_rkvdec2 fdc38100.rkvdec-core: reset done [ 272.882048] mpp_rkvdec2 fdc48100.rkvdec-core: resetting for err 0x23 [ 272.882203] mpp_rkvdec2 fdc48100.rkvdec-core: reset done [ 272.882210] mpp_rkvdec2 fdc38100.rkvdec-core: resetting for err 0x107 [ 272.882332] mpp_rkvdec2 fdc38100.rkvdec-core: reset done And after a few minutes the application freezes again.

[nyanmisaka]

That just means that the video stream you provided contains values ​​that do not meet the codec specifications, or corrupted data, etc. So the decoder is reset to recover.