mpph265enc: AppSink Bitrate changes cause frames to be lost

[MentosGuy]

Hello, I recently observed the following: I use a C application that manages a Gstreamer pipeline as AppSink. Whenever I change the bitrate of the mpph265enc encoder via the AppSink, the encoder loses a few frames that are perceived as microstutters. Is there any way to prevent this? It is a 1080p60 input.

[JeffyCN]

could you provide a demo to reproduce it?

[MentosGuy]

Thanks for the quick reply! I'll put together a demo within the next few days and post it here.

[MentosGuy]

Here I am again! I have a ZIP file with source and finished export file with source code. Can I send it to you directly via direct message or something? Thanks already.

[JeffyCN]

try to upload to google drive and send me a link

[MentosGuy]

I uploaded it to Workupload Password: 75x8JvRYPM1gLUL9

[MentosGuy]

There you can see the lags only minimally, otherwise I can try to record a video with more movement. Motorbike recordings at 100 km/h or more are then more representative, constantly lost frames. But you can see it here too.

[MentosGuy]

The following always appears in the log when the frames are lost: mpp_enc: MPP_ENC_SET_RC_CFG bps 12000000 [11250000 : 12750000] fps [60:60] gop 120 mpp_enc: mode cbr bps [11250000:12000000:12750000] fps fix [60/1] -> fix [60/1] gop i [120] v [0] Is it possible that the encoder is restarted briefly each time the RC Config is reset and therefore frames are missing?

[JeffyCN]

i could not reproduce the issue on my rk3588 evb by checking the input frames and output frames.

diff --git a/gst/rockchipmpp/gstmppdec.c b/gst/rockchipmpp/gstmppdec.c
index 4f5e447..3f51712 100644
--- a/gst/rockchipmpp/gstmppdec.c
+++ b/gst/rockchipmpp/gstmppdec.c
@@ -26,6 +26,8 @@
 #include "config.h"
 #endif

+#include <stdio.h>
+
 #include "gstmppallocator.h"
 #include "gstmppdec.h"

@@ -991,6 +993,7 @@ gst_mpp_dec_loop (GstVideoDecoder * decoder)
   GST_DEBUG_OBJECT (self, "finish frame ts=%" GST_TIME_FORMAT,
       GST_TIME_ARGS (frame->pts));

+  fprintf(stderr, "finish dec frame %d\n", frame->system_frame_number);
   gst_video_decoder_finish_frame (decoder, frame);

 out:
@@ -1048,6 +1051,8 @@ gst_mpp_dec_handle_frame (GstVideoDecoder * decoder, GstVideoCodecFrame * frame)

   GST_DEBUG_OBJECT (self, "handling frame %d", frame->system_frame_number);

+  fprintf(stderr, "handle dec frame %d\n", frame->system_frame_number);
+
   if (G_UNLIKELY (self->flushing))
     goto flushing;

diff --git a/gst/rockchipmpp/gstmppenc.c b/gst/rockchipmpp/gstmppenc.c
index 65fa180..98f8299 100644
--- a/gst/rockchipmpp/gstmppenc.c
+++ b/gst/rockchipmpp/gstmppenc.c
@@ -26,6 +26,7 @@
 #include "config.h"
 #endif

+#include <stdio.h>
 #include <string.h>

 #include "gstmppallocator.h"
@@ -1067,6 +1068,8 @@ gst_mpp_enc_poll_packet_locked (GstVideoEncoder * encoder)
   GST_DEBUG_OBJECT (self, "finish frame ts=%" GST_TIME_FORMAT,
       GST_TIME_ARGS (frame->pts));

+  fprintf(stderr, "finish enc frame %d\n", frame->system_frame_number);
+
   gst_video_encoder_finish_frame (encoder, frame);

 out:
@@ -1122,6 +1125,8 @@ gst_mpp_enc_handle_frame (GstVideoEncoder * encoder, GstVideoCodecFrame * frame)

   GST_DEBUG_OBJECT (self, "handling frame %d", frame->system_frame_number);

+  fprintf(stderr, "handle enc frame %d\n", frame->system_frame_number);
+
   GST_MPP_ENC_LOCK (encoder);

   if (G_UNLIKELY (self->flushing))

there's 3665 frames decoded and 3665 frames encoded. (frame->system_frame_number 0-3664)

[MentosGuy]

Thank you for the quick reply! Do you have framedrops or none at all when playing the Output.ts in the ZIP file? Thanks for the patch, I'll test it to see if it results in the same thing for me. I also use mpp Upstream, so I built the develop branch myself, the latest version. As well as the latest version i built myself from gstreamer-rockchip of this repo. I pulled the kernel from Armbian in December 2023 and last week I also applied all the patches in the drivers/video/rockchip path that have been available in the last 10 months. Would I have to install another kernel / other patches or do you have a different version? To narrow down the error. I also use an RK3588 (Orange Pi 5 Plus) board.

[JeffyCN]

i checked your source and output videos on PC, they also have 3665 video packets: ffprobe -show_packets Output.ts |grep video |wc -l

[MentosGuy]

Thank you for your help! I'll have another look to see if there isn't a problem somewhere.