search

rockchip-linux / mpp

Media Process Platform (MPP) module
513 stars 161 forks source link

Updagrading to latest version of mpp library degrades framerate #154

Open jpiat opened 4 years ago

jpiat commented 4 years ago

Hi,

i was getting 25FPS out of my gstreamer pipeline using an older version of mpp (stock version on my Khadas Edge) and i started having an issue where the bitrate would become a lot higher than the target bitrate (3 times) on dark scene. I switched to the latest release of mpp and re-compiled gstreamer-rockchip but now the framerate is 15FPS with the following message

mpp[5158]: hal_h264e_vepu_v2: warnning: input buffer size 0x3fc000 is smaller then required size 0x7f8000

anything i'm doing wrong ?

jpiat commented 4 years ago

removed the warning from the code (just to make sure the print is not actually causing the delay) and i still get ~16FPS. I can get 25FPS only when switching to 720p.

HermanChen commented 4 years ago

Maybe there is frame buffer copy in the process. Can you use top to check the CPU usage? And use echo 0x100 > /sys/modules/rk_vcodec/parameters/debug to check the hardware timing.

jpiat commented 4 years ago

using the debug command you sent, here is what i get :+1:

[ 161.473333] rk_vcodec: vpu2_enc task: 21 ms

which probably means that the encoder should be able to tackle 30fps ... The test command i''m using is

gst-launch-1.0 -v rkv4l2src num-buffers=-1 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! queue ! mpph264enc ! fpsdisplaysink fps-update-interval=300

the rkv4l2src was tested alone to 25FPS using

gst-launch-1.0 -v rkv4l2src num-buffers=-1 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! queue ! fpsdisplaysink fps-update-interval=300

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 161, dropped: 0, current: 24.99, average: 25.16

is this a gstreamer issue because of buffer copy ? Is there any sample application that uses the isp as an image source and encode, outside of gstreamer ?

HermanChen commented 4 years ago

Yes, it is mainly software difference cause the issue. I am not sure about the up layer sofeware. Need others help.

HermanChen commented 4 years ago

Can you run the mpi_enc_test and compare the performance difference? Then use perf to check the cpu usage.

jpiat commented 4 years ago

Hi,

i added a profile timer to the gstreamer plugin for the mpp and the process buffer method takes between 32ms and 76ms, which i guess is the issue. The mpp lib is not to blame here, but rather the gstreamer support for it. I'll keep digging.

thanks for you help

jpiat commented 4 years ago

in the gstreamer pipeline just pushing the packet to the encoder takes between 32ms and 77ms. Buffer copy is the issue here.

jpiat commented 4 years ago

Ok, after a bit more testing with using a videotest instead of a camera source, the encoder does 25FPS not problem. So it means that the main issue is the rkv4l2src plugin and not the encoder itself.

jpiat commented 4 years ago

Hi,

in order to test the mpp encoder outside of gstreamer, i made my own code to grab video frame using v4l2 and encode using the mpp library. The frame grabbing is confirmed to be done at 30FPS (with not encoder), the encoding time per frame (input buffer set at 0) is confirmed to be 25ms but when using the encoder with te v4l2 input (1080p@30fps) the framerate varies between 30Fps to 10Fps with an average of 17Fps (over a 100sec period). If found two issues :

jpiat commented 3 years ago

I'm still stuck with this issue of not being able to encode at 25/30FPS when using a V4L2 source. Did anyone succeed to achive 30FPS H264 encoding for a v4L2 source (CSI camera outputting YUYV) ? I'd be happy to share my code with anyone wanting to reproduce the issue.

HermanChen commented 3 years ago

Can you use mpp_time to check the input frame rate from V4L2 to mpp encoder?

jpiat commented 3 years ago

Hi,

when using the v4l2 capture and then encoding, the input frame-rate to video encoder is variable (between 7FPS to 20FPS) but when using the v4l2 capture alone (no encoding on mpp) the frame rate is stable at 30FPS. Pushing to the mpp encoder and then getting the resulting buffer is stable at 25ms, but using the mpp encoder results in the frame rate dropping on the v4l2 capture. The v4L2 input that I'm using is a CSI camera and the v4l2 driver provided by rockchip taps into the ISP. My concern here is that there is an interaction between the ISP driver and the MPP encoder that prevents achieving max frame-rate. In the mpp code i see that there are blank functions to encode from ISP so i guess that it would be far easier to have the MPP use ISP buffer rather than having to go back to user-space in-between.

One can test my code at

https://github.com/jpiat/mpp_encoder_test

it works for a 1080p camera outputting in YUYV format connected to one of the CSI interface (http://wiki.veye.cc/index.php/VEYE-MIPI-290/327) connected to a Khadas Edge board. Same code should work with any other rk3399 board.