search

Freescale / gstreamer-imx

GStreamer 1.0 plugins for i.MX platforms
Other
183 stars 127 forks source link

imxvpuenc_h264 does not output regular PPS? #234

Closed kumsaati1 closed 2 years ago

kumsaati1 commented 5 years ago

Hello, I have the pipeline: appsrc name=mysource format=3 is-live=true ! imxvpuenc_h264 idr-interval=16 ! video/x-h264,alignment=au ! rtph264pay ! udpsink which I feed its source in the push mode. However, I have a problem in receiving it, and the rtph264depay plugin at the other end spits out insane PPS parameters, which I believe the cause of the problem. When I debug the server pipeline above, I have realized that rtph264pay does not understand the limit of the PPS and packs the stream data along with the PPS. The below debug trace explains my problem better:

numOfSlices 1 0:00:01.511217334 667 0x542200 LOG imxvpuapi imxvpuapi_vpulib.c:3986:imx_vpu_enc_encode: added h.264 AUD 0:00:01.511319334 667 0x542200 LOG imxvpuapi imxvpuapi_vpulib.c:2848:imx_vpu_enc_write_header_data: added h.264 SPS RBSP with 14 byte 0:00:01.511409667 667 0x542200 LOG imxvpuapi imxvpuapi_vpulib.c:2849:imx_vpu_enc_write_header_data: added h.264 PPS RBSP with 8 byte 0:00:01.516623334 667 0x542200 LOG imxvpuapi imxvpuapi_vpulib.c:4034:imx_vpu_enc_encode: added main encoded frame data with 347866 byte 0:00:01.517054001 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:1332:gst_rtp_h264_pay_sink_event: New stream detected => Clear SPS and PPS 0:00:01.517898334 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:345:gst_rtp_h264_pay_getcaps: Intersect video/x-h264, profile=(string)constrained-baseline, stream-format=(string)avc, alignment=(string)au; video/x-h264, stream-format=(string)avc, alignment=(string)au; video/x-h264, profile=(string)constrained-baseline, stream-format=(string)byte-stream, alignment=(string){ nal, au }; video/x-h264, stream-format=(string)byte-stream, alignment=(string){ nal, au } and filter video/x-h264, alignment=(string)au, stream-format=(string)byte-stream, parsed=(boolean)true, width=(int)1280, height=(int)1024, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:0:0 0:00:01.518557334 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:345:gst_rtp_h264_pay_getcaps: Intersect video/x-h264, profile=(string)constrained-baseline, stream-format=(string)avc, alignment=(string)au; video/x-h264, stream-format=(string)avc, alignment=(string)au; video/x-h264, profile=(string)constrained-baseline, stream-format=(string)byte-stream, alignment=(string){ nal, au }; video/x-h264, stream-format=(string)byte-stream, alignment=(string){ nal, au } and filter video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, parsed=(boolean)true, width=(int)1280, height=(int)1024, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:0:0 0:00:01.518917001 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:566:gst_rtp_h264_pay_setcaps: have bytestream h264 0:00:01.519724001 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:1063:gst_rtp_h264_pay_handle_buffer: got 347894 bytes (347894) 0:00:01.519847667 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:1147:gst_rtp_h264_pay_handle_buffer: found first start at 1, bytes left 347893 0:00:01.519928334 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:1176:gst_rtp_h264_pay_handle_buffer: found next start at 3 of size 3 0:00:01.519993667 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:694:gst_rtp_h264_pay_decode_nal: NAL payload len=3 0:00:01.520058334 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:719:gst_rtp_h264_pay_decode_nal: NAL: 0 0 9 Len = 3 0:00:01.520130001 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:1176:gst_rtp_h264_pay_handle_buffer: found next start at 11 of size 11 0:00:01.520192667 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:694:gst_rtp_h264_pay_decode_nal: NAL payload len=11 0:00:01.520255001 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:706:gst_rtp_h264_pay_decode_nal: Found SPS 0 3 7 Len=11 0:00:01.522074667 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:679:next_start_code: Cannot find next NAL start code. returning 347870 0:00:01.522225334 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:1176:gst_rtp_h264_pay_handle_buffer: found next start at 347870 of size 347870 0:00:01.522313667 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:694:gst_rtp_h264_pay_decode_nal: NAL payload len=347870 0:00:01.522390334 667 0x542200 DEBUG rtph264pay gstrtph264pay.c:706:gst_rtp_h264_pay_decode_nal: Found PPS 0 3 8 Len=347870

As you can see the imxvpulib says it added a 8 byte regular PPS parameter, but rtph264pay cannot see that it is 8 bytes and thinks the whole buffer is the PPS parameter. I think this might be caused in the encoder plugin, either in the encoding process, or the packing of it in the buffers. Weirdly, this does not happen if I use videotestsrc with same heigth/width and format, instead of appsrc. Is there a solution for this problem?

Thank you.

angolini commented 3 years ago

Could you, please, test it again using the latest release and update the status of this issue, @kumsaati1 ?