Multiple encoders in pipeline

[sergeyKviat]

We use gstreamer to create hls streams with adaptive bitrate. Currently video are transcoded using elements such as videorate, videoscale and x264enc. It works fine, but we'd like to switch to MFX elements to reduce CPU load and increase performance so we try to use following pipeline:

gst-launch-1.0 --gst-debug=mfx*:5 souphttpsrc location=http://tva.fastbroad.com/i/albinfobx1_1@388038/master.m3u8 ! hlsdemux ! tsdemux ! multiqueue ! h264parse ! mfxh264dec ! tee name=t ! queue ! mfxvpp width=800 height=600 ! mfxh264enc ! mpegtsmux ! hlssink location="1segment%05d.ts" playlist-location="1playlist.m3u8" t. ! queue ! mfxvpp width=640 height=480 ! mfxh264enc ! mpegtsmux ! hlssink location="2segment%05d.ts" playlist-location="2playlist.m3u8"

It crashs with following messages:

(gst-plugin-scanner:18752): GLib-GObject-CRITICAL **: g_object_class_install_properties: assertion 'pspec != NULL' failed 0:00:00.121690417 0x22b0030 DEBUG mfx gstmfxdisplay.c:347:gst_mfx_display_new: creating dummy display libva info: VA-API version 0.99.0 libva info: va_getDriverName() returns 0 libva info: User requested driver 'iHD' libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so libva info: Found init function __vaDriverInit_0_32 libva info: va_openDriver() returns 0 0:00:00.122986337 0x22b0030 DEBUG mfx gstmfxdisplay.c:461:gst_mfx_display_init_vaapi: VA-API version 0.99 0:00:00.474230064 0x7fc910015cf0 INFO mfxdecode gstmfxdec.c:189:gst_mfxdec_update_sink_caps: new sink caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, framerate=(fraction)50/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)high, level=(string)3.2, codec_data=(buffer)01640020ffe1001867640020acd9405005ba10000003001000000648f183196001000568ebecb22c 0:00:00.474644524 0x7fc910015cf0 INFO mfxdecode gstmfxdec.c:231:gst_mfxdec_update_src_caps: new src caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1 0:00:00.479748547 0x7fc910015cf0 INFO mfx gstmfxtaskaggregator.c:158:gst_mfx_task_aggregator_create_session: Using Media SDK API version 1.21 0:00:00.479791819 0x7fc910015cf0 INFO mfx gstmfxtaskaggregator.c:176:gst_mfx_task_aggregator_create_session: Initialized internal MFX session using hardware accelerated implementation 0:00:00.480006118 0x7fc910015cf0 DEBUG mfxdecode gstmfxdec.c:247:gst_mfxdec_negotiate: Input codec state changed, doing renegotiation 0:00:00.480372611 0x7fc910015cf0 INFO mfxdecode gstmfxdec.c:231:gst_mfxdec_update_src_caps: new src caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1

(gst-launch-1.0:18751): GStreamer-CRITICAL **: gst_debug_log_valist: assertion 'category != NULL' failed 0:00:00.481198745 0x23f40a0 INFO mfxvpp gstmfxpostproc.c:514:gst_mfxpostproc_update_sink_caps: new sink caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1 0:00:00.481238581 0x23f41e0 INFO mfxvpp gstmfxpostproc.c:514:gst_mfxpostproc_update_sink_caps: new sink caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1 0:00:00.481304283 0x23f40a0 INFO mfxvpp gstmfxpostproc.c:486:gst_mfxpostproc_update_src_caps: new src caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)800, height=(int)600, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1 0:00:00.481332677 0x23f41e0 INFO mfxvpp gstmfxpostproc.c:486:gst_mfxpostproc_update_src_caps: new src caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)50/1 0:00:00.484877744 0x7fc910015cf0 INFO mfx gstmfxdecoder.c:746:gst_mfx_decoder_start: Initialized MFX decoder task using video memory 0:00:00.485367226 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -10 0:00:00.485823321 0x23f40a0 INFO mfx gstmfxtaskaggregator.c:158:gst_mfx_task_aggregator_create_session: Using Media SDK API version 1.21 0:00:00.485866452 0x23f40a0 INFO mfx gstmfxtaskaggregator.c:176:gst_mfx_task_aggregator_create_session: Initialized internal MFX session using hardware accelerated implementation 0:00:00.485904222 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -10 0:00:00.488815717 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -11 0:00:00.488872761 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -10 0:00:00.489296732 0x23f41e0 INFO mfx gstmfxtaskaggregator.c:158:gst_mfx_task_aggregator_create_session: Using Media SDK API version 1.21 0:00:00.489339176 0x23f41e0 INFO mfx gstmfxtaskaggregator.c:176:gst_mfx_task_aggregator_create_session: Initialized internal MFX session using hardware accelerated implementation 0:00:00.491029583 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -11 0:00:00.491081019 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -10 0:00:00.496641991 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -11 0:00:00.496713621 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -10 0:00:00.498281818 0x23f40a0 DEBUG mfxvpp gstmfxpostproc.c:958:gst_mfxpostproc_query: sharing tasks 0x22af330 0:00:00.502130074 0x23f40a0 INFO mfx gstmfxtaskaggregator.c:158:gst_mfx_task_aggregator_create_session: Using Media SDK API version 1.21 0:00:00.502181054 0x23f40a0 INFO mfx gstmfxtaskaggregator.c:176:gst_mfx_task_aggregator_create_session: Initialized internal MFX session using hardware accelerated implementation 0:00:00.502532859 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -11 0:00:00.502583903 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: -10 0:00:00.502606857 0x23f40a0 INFO mfxh264enc gstmfxenc_h264.c:178:gst_mfxenc_h264_set_config: using high profile as target decoder constraints 0:00:00.502638704 0x23f40a0 DEBUG mfx gstmfxencoder_h264.c:124:gst_mfx_encoder_h264_reconfigure: resolution: 800x600 0:00:00.502797718 0x23f40a0 WARN mfx gstmfxencoder.c:940:gst_mfx_encoder_start: Incompatible video params detected 5 0:00:00.503493352 0x23f41e0 DEBUG mfxvpp gstmfxpostproc.c:958:gst_mfxpostproc_query: sharing tasks 0x22af330 0:00:00.503561990 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: 0 0:00:00.503587490 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1110:gst_mfx_decoder_decode: MFXVideoCORE_SyncOperation status: 0 0:00:00.503634232 0x23f41e0 INFO mfxh264enc gstmfxenc_h264.c:178:gst_mfxenc_h264_set_config: using high profile as target decoder constraints 0:00:00.503652050 0x23f41e0 DEBUG mfx gstmfxencoder_h264.c:124:gst_mfx_encoder_h264_reconfigure: resolution: 640x480 0:00:00.504734036 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: 0 0:00:00.504762128 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1110:gst_mfx_decoder_decode: MFXVideoCORE_SyncOperation status: 0 0:00:00.510270389 0x23f40a0 INFO mfx gstmfxencoder.c:1024:gst_mfx_encoder_start: Initialized MFX encoder task using input video memory surfaces 0:00:00.510350955 0x23f40a0 DEBUG mfxvpp gstmfxpostproc.c:958:gst_mfxpostproc_query: sharing tasks 0x22af330 0:00:00.510567660 0x23f40a0 INFO mfx gstmfxfilter.c:1052:gst_mfx_filter_start: Initialized MFX VPP output task using video memory 0:00:00.512619445 0x23f41e0 INFO mfx gstmfxencoder.c:1024:gst_mfx_encoder_start: Initialized MFX encoder task using input video memory surfaces 0:00:00.512666866 0x23f41e0 DEBUG mfxvpp gstmfxpostproc.c:958:gst_mfxpostproc_query: sharing tasks 0x22af330 0:00:00.512833351 0x23f41e0 INFO mfx gstmfxfilter.c:1052:gst_mfx_filter_start: Initialized MFX VPP output task using video memory 0:00:00.513371175 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: 0 0:00:00.513390366 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1110:gst_mfx_decoder_decode: MFXVideoCORE_SyncOperation status: 0 0:00:00.513799880 0x23f41e0 ERROR mfx gstmfxencoder.c:1092:gst_mfx_encoder_encode: Error during MFX encoding. 0:00:00.513814365 0x23f41e0 ERROR mfxencode gstmfxenc.c:448:gst_mfxenc_handle_frame: failed to encode frame 0 (status -1) 0:00:00.514122242 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1054:gst_mfx_decoder_decode: MFXVideoDECODE_DecodeFrameAsync status: 0 0:00:00.514329926 0x7fc910015cf0 DEBUG mfx gstmfxdecoder.c:1110:gst_mfx_decoder_decode: MFXVideoCORE_SyncOperation status: 0 ERROR: from element /GstPipeline:pipeline0/GstHLSDemux:hlsdemux0: Internal data stream error. Additional debug info: gstadaptivedemux.c(2531): _src_chain (): /GstPipeline:pipeline0/GstHLSDemux:hlsdemux0: streaming stopped, reason error (-5) ERROR: pipeline doesn't want to preroll. Caught SIGSEGV

Sometimes it works for a few seconds or even minutes, but then freeze or crash with segfault anyway. Also produced stream is completely unwatchable. We aware of threading conflicts between gstreamer and mfx and tried different values of async-depth property of mfx elements, but it doesn't help.

gstreamer-media-SDK version: 1.4.0.3

[ishmael1985]

This issue has been resolved in my fork as well as in the topic_linux_and_window branch

https://github.com/ishmael1985/gstreamer-media-SDK https://github.com/intel/gstreamer-media-SDK/tree/topic_linux_and_window

Try the latest source from the above and let us know if it resolves your issue.

[ishmael1985]

@sergeyKviat is this issue resolved for you? Please close when it is.

[sergeyKviat]

I'm sorry for late response. Issue was resolved when we switched to topic_linux_and_window branch and changed pipeline as described in README.USAGE file. Thanks for your help.