[gstreamer-rockchip]Rendering on Android consume a lot of CPU

[Meonardo]

Env:

• Kernel info: Linux localhost 5.10.66 #394 SMP PREEMPT Fri Jul 7 13:30:47 CST 2023 aarch64;
• GStreamer v1.22.6;
• gstreamer-rockchip latest code;
• single H.264 RTSP video stream(1920x1080@25fps)

---

GStreamer pipelines:

1. without renderer, works great low CPU usage, very fast at decoding.

   rtspsrc location=%s ! rtph264depay ! h264parse ! mppvideodec ! fakesink

   

2. with renderer(glimagesink), the CPU up to 105%

   rtspsrc location=%s ! rtph264depay ! h264parse ! mppvideodec ! glimagesink

   

---

I am using SurfaceView as renderer on Android, in native I got the ANativeWindow from ANativeWindow_fromSurface(env, surface), then I use gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), (guintptr)window) pass to GStreamer. Did I miss something? Do I need apply some patches in GStreamer? @JeffyCN sorry to bother you, could you give me some advice? thanks.

[ScorpiosCrux]

I have the same issue by the way. Even on an RK3588, decoding anything higher than 1080p 60FPS, causes the CPU to go extremely high.

[JeffyCN]

for glimagesink, it will try to import buffer to texture, which is normally a neon version of memcpy(inside of the gl texture2d impl).

check the gl related hack patches under: https://github.com/JeffyCN/rockchip_mirrors/tree/buildroot/package/gstreamer1/gst1-plugins-base

for dmabuf direct importing

[Meonardo]

After applied the patch 1 and patch 2 to GStreamer, I have managed to low the CPU usage down to 40% for a single H.264 RTSP video(1920x1080@25fps) decoding & rendering, does it look normal to you on an rk3588? @JeffyCN, thank you for your reply!

[JeffyCN]

i haven't use android with this before, you can try to check the cpu usage of threads, the decoder and renderer should not cause much with these patches.