search

JeffyCN / rockchip_mirrors

Mirrors of a few Rockchip BSP repositories, some others in https://github.com/JeffyCN/mirrors.
Other
10 stars 2 forks source link

[RK3588] Single gstreamer pipeline not able to hit 4K@120 fps, but two separate pipelines hit 4K@60 fps each #21

Closed lgarbarini closed 9 months ago

lgarbarini commented 1 year ago

I have a gstreamer pipeline either pulling for USB/MIPI/file I expect to be able to achieve 4K@120 FPS. However, I seem to be unable to get a single encoder to achieve 4K@120 fps. However, two pipelines at the same time are able to achieve >120 fps combined.

As videotestsrc isn't able to keep up on the RK3588, I run it at a slower framerate and duplicate frames with videorate for the purposes of a consistent input. I have tried with both live and test sources and the results seem to be the same.

EDIT: See below, but it looks like only a single rkvenc-core is being used in the 4K@120 fps case limiting it to 4K@60 fps. Is it possible to utilize the second core for 4K@120fps?

8k@30 fps, works as expected (~29 fps average):

gst-launch-1.0 videotestsrc ! video/x-raw,framerate=6/1 ! videorate ! queue ! video/x-raw,width=7680,height=4320,framerate=60/1,format=NV12 ! mpph265enc ! fpsdisplaysink video-sink=fakesink text-overlay=false -ve

1080P@240 fps, works as expected (~230 fps average):

gst-launch-1.0 videotestsrc ! video/x-raw,framerate=30/1 ! videorate ! queue ! video/x-raw,width=1920,height=1080,framerate=240/1,format=NV12 ! mpph265enc ! fpsdisplaysink video-sink=fakesink text-overlay=false -ve

4K@120 fps, Doesn't work as expected (63 fps average):

gst-launch-1.0 videotestsrc ! video/x-raw,framerate=30/1 ! videorate ! queue ! video/x-raw,width=3840,height=2160,framerate=120/1,format=NV12 ! mpph265enc ! fpsdisplaysink video-sink=fakesink text-overlay=false -ve

However, if I run 2x of the above 4K pipelines simultaneously, both achieve 4K@60 fps (120 fps total):

# In terminal 1:
gst-launch-1.0 videotestsrc ! video/x-raw,framerate=30/1 ! videorate ! queue ! video/x-raw,width=3840,height=2160,framerate=120/1,format=NV12 ! mpph265enc ! fpsdisplaysink video-sink=fakesink text-overlay=false -ve
# results in ~63 fps average

# In terminal 2:
gst-launch-1.0 videotestsrc ! video/x-raw,framerate=30/1 ! videorate ! queue ! video/x-raw,width=3840,height=2160,framerate=120/1,format=NV12 ! mpph265enc ! fpsdisplaysink video-sink=fakesink text-overlay=false -ve
# results in ~63 fps average
image

Same issue occurs using filesrc (60fps from 120fps source mkv):

gst-launch-1.0 filesrc location=/tmp/test.mkv ! matroskademux ! h265parse ! mppvideodec ! queue ! video/x-raw ! mpph265enc ! fpsdisplaysink video-sink=fakesink text-overlay=false -ve

Other system information:

dmesg output with debug on when running 2x4K@60fps (120 fps total):

[22140.018810] rk_vcodec: fdbd0000.rkvenc-core:0 session 18890:1027 time: 14574 us
[22140.018976] rk_vcodec: fdbe0000.rkvenc-core:1 session 18899:1031 time: 14595 us
[22140.033929] rk_vcodec: fdbd0000.rkvenc-core:0 session 18890:1027 time: 14627 us
[22140.034083] rk_vcodec: fdbe0000.rkvenc-core:1 session 18899:1031 time: 14565 us
[22140.049023] rk_vcodec: fdbd0000.rkvenc-core:0 session 18890:1027 time: 14640 us
[22140.051221] rk_vcodec: fdbe0000.rkvenc-core:1 session 18899:1031 time: 15537 us
[22140.064179] rk_vcodec: fdbd0000.rkvenc-core:0 session 18890:1027 time: 14634 us
[22140.066364] rk_vcodec: fdbe0000.rkvenc-core:1 session 18899:1031 time: 14640 us
[22140.081486] rk_vcodec: fdbe0000.rkvenc-core:1 session 18899:1031 time: 14572 us
[22140.082055] rk_vcodec: fdbd0000.rkvenc-core:0 session 18890:1027 time: 15549 us
[22140.097545] rk_vcodec: fdbd0000.rkvenc-core:0 session 18890:1027 time: 14587 us
[22140.097688] rk_vcodec: fdbe0000.rkvenc-core:1 session 18899:1031 time: 14569 us

dmesg output with debug on when running 1x4K@120fps (~63fps actual):

[22874.355580] rk_vcodec: fdbd0000.rkvenc-core:0 session 18951:1039 time: 14637 us
[22874.370727] rk_vcodec: fdbd0000.rkvenc-core:0 session 18951:1039 time: 14629 us
[22874.386210] rk_vcodec: fdbd0000.rkvenc-core:0 session 18951:1039 time: 14717 us
[22874.403979] rk_vcodec: fdbd0000.rkvenc-core:0 session 18951:1039 time: 15639 us

dmesg output with debug on when running 1x8k@30fps:

[22999.977358] rk_vcodec: fdbe0000.rkvenc-core:1 session 18970:1043 time: 29922 us
[23000.007269] rk_vcodec: fdbd0000.rkvenc-core:0 session 18970:1043 time: 29127 us
[23000.008150] rk_vcodec: fdbe0000.rkvenc-core:1 session 18970:1043 time: 29854 us
[23000.163325] rk_vcodec: fdbd0000.rkvenc-core:0 session 18970:1043 time: 29207 us
[23000.164595] rk_vcodec: fdbe0000.rkvenc-core:1 session 18970:1043 time: 30309 us
lgarbarini commented 1 year ago

Looks like setting auto_tile to 1 enables the use of both cores.

https://github.com/JeffyCN/rockchip_mirrors/blob/546cb913e3feabc4a41a6220fa80d81e0dea71c2/gst/rockchipmpp/gstmpph265enc.c#L185

diff --git a/gst/rockchipmpp/gstmpph265enc.c b/gst/rockchipmpp/gstmpph265enc.c
index 2605f0f..4ce315e 100644
--- a/gst/rockchipmpp/gstmpph265enc.c
+++ b/gst/rockchipmpp/gstmpph265enc.c
@@ -182,6 +182,7 @@ gst_mpp_h265_enc_apply_properties (GstVideoEncoder * encoder)
   if (G_LIKELY (!mppenc->prop_dirty))
     return TRUE;

+  mpp_enc_cfg_set_s32 (mppenc->mpp_cfg, "h265:auto_tile", 1);
   mpp_enc_cfg_set_s32 (mppenc->mpp_cfg, "h265:qp_init", self->qp_init);

   if (mppenc->rc_mode == MPP_ENC_RC_MODE_FIXQP) {
JeffyCN commented 9 months ago

not sure about the "auto_tile" @HermanChen but i've uploaded a patch in gst-mpp for async encoding.