gstreamer hangs after a while playing hevc 10bit and av1 10bit

avafinger commented 1 year ago

Hi @JeffyCN ,

I am trying to decode hevc 10-bit and av1 10-bit, it works for a while and then gives an internal error, exactly at the same offset anytime i play the file.

You can reproduce the error with:

HEVC 10bit:

gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-20-mbps-hd-hevc-10bit.mkv ! matroskademux ! h265parse ! mppvideodec ! videoconvert ! video/x-raw,format=NV12_10LE40 ! appsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
Execution ended after 0:00:19.272551234
Setting pipeline to NULL ...
Freeing pipeline ...

verbose:

gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-20-mbps-hd-hevc-10bit.mkv ! matroskademux ! h265parse ! mppvideodec ! videoconvert ! video/x-raw,format=NV12_10LE40 ! appsink -v
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, level=(string)4, tier=(string)high, profile=(string)main-10, codec_data=(buffer)01222000000090000000000078f000fcfdfafa00000f03200001001840010c01ffff222000000300900000030000030078999809210001002a420101222000000300900000030000030078a003c08010e4d96666924caf0101000003006400000bb50822000100074401c172b46240, stream-format=(string)hvc1, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30000/1001, chroma-format=(string)4:2:0, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstMppVideoDec:mppvideodec0.GstPad:sink: caps = video/x-h265, level=(string)4, tier=(string)high, profile=(string)main-10, codec_data=(buffer)01222000000090000000000078f000fcfdfafa00000f03200001001840010c01ffff222000000300900000030000030078999809210001002a420101222000000300900000030000030078a003c08010e4d96666924caf0101000003006400000bb50822000100074401c172b46240, stream-format=(string)hvc1, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30000/1001, chroma-format=(string)4:2:0, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:sink: caps = video/x-h265, level=(string)4, tier=(string)high, profile=(string)main-10, codec_data=(buffer)01222000000090000000000078f000fffdfafa00000f03a00001001840010c01ffff222000000300900000030000030078999809a10001002a420101222000000300900000030000030078a003c08010e4d96666924caf0101000003006400000bb508a2000100074401c172b46240, stream-format=(string)hvc1, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30000/1001
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, level=(string)4, tier=(string)high, profile=(string)main-10, codec_data=(buffer)01222000000090000000000078f000fcfdfafa00000f03200001001840010c01ffff222000000300900000030000030078999809210001002a420101222000000300900000030000030078a003c08010e4d96666924caf0101000003006400000bb50822000100074401c172b46240, stream-format=(string)hvc1, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30000/1001, chroma-format=(string)4:2:0, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstMppVideoDec:mppvideodec0.GstPad:sink: caps = video/x-h265, level=(string)4, tier=(string)high, profile=(string)main-10, codec_data=(buffer)01222000000090000000000078f000fcfdfafa00000f03200001001840010c01ffff222000000300900000030000030078999809210001002a420101222000000300900000030000030078a003c08010e4d96666924caf0101000003006400000bb50822000100074401c172b46240, stream-format=(string)hvc1, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30000/1001, chroma-format=(string)4:2:0, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstMppVideoDec:mppvideodec0.GstPad:src: caps = video/x-raw(memory:DMABuf), format=(string)NV12_10LE40, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw(memory:DMABuf), format=(string)NV12_10LE40, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12_10LE40, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001
/GstPipeline:pipeline0/GstAppSink:appsink0.GstPad:sink: caps = video/x-raw, format=(string)NV12_10LE40, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:DMABuf), format=(string)NV12_10LE40, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw(memory:DMABuf), format=(string)NV12_10LE40, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
Execution ended after 0:00:19.268115245
Setting pipeline to NULL ...
Freeing pipeline ...

AV1 10bit:

gst-launch-1.0 filesrc location=/apps/Videos/av1-10bit-1080p-sample.mkv ! matroskademux ! av1parse ! mppvideodec ! videoconvert ! video/x-raw,format=NV12_10LE40 ! appsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
Execution ended after 0:00:23.360245123
Setting pipeline to NULL ...
Freeing pipeline ...

avafinger commented 1 year ago

Additional info:


0:00:19.394285934 15753   0x7fb4067460 ERROR                 mppdec gstmppdec.c:1012:gst_mpp_dec_handle_frame:<mppvideodec0> failed to send packet
0:00:19.395494857 15753   0x7fb4067460 WARN                  mppdec gstmppdec.c:1016:gst_mpp_dec_handle_frame:<mppvideodec0> can't handle this frame
0:00:19.396545702 15753   0x5567d58de0 WARN           matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: Internal data stream error.
0:00:19.397677044 15753   0x5567d58de0 WARN           matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: streaming stopped, reason error (-5)
0:00:19.398770179 15753   0x5567d58de0 INFO        GST_ERROR_SYSTEM gstelement.c:2271:gst_element_message_full_with_details:<matroskademux0> posting message: Internal data stream error.
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
0:00:19.405222826 15753   0x5567d58de0 INFO        GST_ERROR_SYSTEM gstelement.c:2298:gst_element_message_full_with_details:<matroskademux0> posted error message: Internal data stream error.
Execution ended after 0:00:19.301138668
Setting pipeline to NULL ...
0:00:19.407927956 15753   0x5567b80e00 INFO              GST_STATES gstbin

avafinger commented 1 year ago

Not sure if it is related but i have this patch applied: https://raw.githubusercontent.com/JeffyCN/rockchip_mirrors/buildroot/package/gstreamer1/gst1-plugins-good/0009-matroska-demux-parse-Disable-QoS-lace-skipping-by-de.patch

rk3588, gstreamer 1.20.3 with all patches applied. FFmpeg play these files just fine, so mpp is working.

JeffyCN commented 1 year ago

looks like the mpp's put packet hang.

maybe try to increase the input timeout: -#define MPP_INPUT_TIMEOUT_MS 2000 / Block timeout for MPP input queue / +#define MPP_INPUT_TIMEOUT_MS 2000000 / Block timeout for MPP input queue /

and use gdb to debug it.

avafinger commented 1 year ago

I built your latest gstreamer-rockchip, still has the same error:

0:00:19.446961334 26242   0x7f88066460 ERROR                 mppdec gstmppdec.c:1036:gst_mpp_dec_handle_frame:<mppvideodec0> failed to send packet
0:00:19.447033081 26242   0x7f88066460 WARN                  mppdec gstmppdec.c:1040:gst_mpp_dec_handle_frame:<mppvideodec0> can't handle this frame
0:00:19.447211284 26242   0x7f88066460 INFO                    task gsttask.c:368:gst_task_func:<multiqueue0:src_0> Task going to paused
0:00:19.447313365 26242   0x55923d7de0 WARN           matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: Internal data stream error.
0:00:19.447345739 26242   0x55923d7de0 WARN           matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: streaming stopped, reason error (-5)

I increased the values of the timeout_ms (5000) and interval_ms (10), same error.

i saw you reply now

avafinger commented 1 year ago

Comparing this to FFmpeg, in which there is no timeout on decode_put_packet, it assumes "Decoder buffer full" and return EAGAIN and then you comment: "some streams might need more packets to start returning frame..." and get next frame.

I don't see this in gstreamer-rockchip, isn't that the case?

JeffyCN commented 1 year ago

ffmpeg and gst are different, i don't think that would be related.

please increase the timeout to confirm if it hang in the put packet, so that we could use gdb to debug it

avafinger commented 1 year ago

please increase the timeout to confirm if it hang in the put packet, so that we could use gdb to debug it

Yes, it hangs. Unfortunately, i don't know how to debug gstreamer with gdb (i know gdb basics). Is there a way to build gstreamer-rockchip with debugging (without stripping and optimizations)? I might debug it using Geany (which uses gdb, :) ) .

ffmpeg and gst are different

Yeah, but a see a pattern:

gstreamer:

mpp_packet_set_pts()
decode_put_packet (); -> if error on timeout, return error
decode_get_frame();

ffmpeg:

ret = decode_get_frame();
        return decode_put_packet ();
if (ret != MPP_OK)
  return (EAGAIN)
decode_get_frame();

Redistribute latency... New clock: GstSystemClock 0:00:16.6 / 0:00:30.0 (55.2 %)

Hang at 00:16.6 every time i run.

gst-launch-1.0 filesrc location=jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! appsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
klass->send_mpp_packet() rc: 0 - timed out with error
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
Execution ended after 0:01:12.808293867
Setting pipeline to NULL ...
Freeing pipeline ...

file: wget http://www.larmoire.info/jellyfish/media/jellyfish-10-mbps-hd-hevc-10bit.mkv

avafinger commented 1 year ago

Additional info, this can decode and display on X11 just fine:


DISPLAY=:0.0 gst-launch-1.0 filesrc location=~/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! xvimagesink

Decode but can't display (black screen)


DISPLAY=:0.0 gst-launch-1.0 filesrc location=~/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! rkximagesink

Crashed:


DISPLAY=:0.0 gst-launch-1.0 filesrc location=~/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! glimagesink
Setting pipeline to PAUSED ...
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
**
GStreamer-GL:ERROR:../gst-libs/gst/gl/gstglformat.c:234:gst_gl_format_from_video_info: code should not be reached
Bail out! GStreamer-GL:ERROR:../gst-libs/gst/gl/gstglformat.c:234:gst_gl_format_from_video_info: code should not be reached
Aborted (core dumped)

and these pipelines do not work (wayland/weston):

gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-20-mbps-hd-hevc-10bit.mkv ! matroskademux ! h265parse ! mppvideodec ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.001810498
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.004942712
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.001950491
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! kmssink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: GStreamer encountered a general resource error.
Additional debug info:
../sys/kms/gstkmssink.c(2063): gst_kms_sink_show_frame (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
drmModeSetPlane failed: Invalid argument (22)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! kmssink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: GStreamer encountered a general resource error.
Additional debug info:
../sys/kms/gstkmssink.c(2063): gst_kms_sink_show_frame (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
drmModeSetPlane failed: Invalid argument (22)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$

JeffyCN commented 1 year ago

1/ if it hang in put packet, then add logs to get frame, if it hang there too, it's an mpp issue

2/ nv12 10 might need more patches in the sink, please try to get 1.20 version patches here: https://github.com/JeffyCN/rockchip_mirrors/commit/9ee48e73ec313323df20f3e69c964dfe46bb0fb4

JeffyCN commented 1 year ago

it works well on my 3588 evb with buildroot sdk: root@rk3588:/# killall weston root@rk3588:/# gst-play-1.0 /jellyfish-10-mbps-hd-hevc-10bit.mkv --videosink=kmssink 按“k”键来显示键盘快捷键列表。正在播放 /jellyfish-10-mbps-hd-hevc-10bit.mkvRedistribute latency... Redistribute latency... Redistribute latency... 0:00:30.0 / 0:00:30.0 到达播放列表结尾。

JeffyCN commented 1 year ago

FYI: 1/ i'm using mpp at: https://github.com/JeffyCN/mirrors/tree/mpp-dev 2/ the gst 1.20 patches at https://github.com/JeffyCN/rockchip_mirrors/commit/9ee48e73ec313323df20f3e69c964dfe46bb0fb4 doesn't contain the position patches(glimagesink and xvimagesink), you may need to add those manually 3/ for waylandsink, you may need to set env "export GST_MPP_DEC_DMA_FEATURE=1" to make waylandsink use dmabuf 4/ for kmssink, you may need to set plane-id property to an non-cluster overlay plane for non-afbc output formats, and maybe configure the plane's zpos property with modetest tool 5/ for rkximagesink, same as kmssink

avafinger commented 1 year ago

Thanks for the info,

I think i have that patches, i will revise . I am particularly interested in appsink so i can use gstreamer programmatically.

Can you please run this pipeline and report the result? gst-launch-1.0 filesrc location=jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! appsink

JeffyCN commented 1 year ago

you may need to write a real appsink to use it, otherwise it will no free the output frames/buffers by default, check it's max-buffers and drop properties.

or please test it with other existing sink.

avafinger commented 1 year ago

or please test it with other existing sink.

@JeffyCN

I already did that and I have the same problem.

will no free the output frames/buffers by default, check it's max-buffers and drop properties.

Trying to figure out how to do this.

max-buffers = 2 No idea about the drop properties, can you give some hints?

So, that means these pipelines:

gst-launch-1.0 filesrc location=jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! appsink

and

gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-20-mbps-hd-hevc-10bit.mkv ! matroskademux ! h265parse ! mppvideodec ! appsink

are expected to fail for the above reason?

JeffyCN commented 1 year ago

1/ as replied before, the waylandsink glimagesink kmssink rkximagesink xvimagesink fakesink both works(some of them needs to set env or property)

2/ appsink drop=true max-buffers=1 'gst-inspect-1.0 appsink' to check the meaning of those properties, basically is to only keep 1 buffer in queue and drop the old buffers.

otherwise it will hold buffers and not to free them, for example 'gst-launch-1.0 videotestsrc ! appsink' you can see the memory usage keep increasing.

avafinger commented 1 year ago

appsink drop=true max-buffers=1

Works, thank you.

JeffyCN commented 1 year ago

guessing the increasing mpp bufs holding too much fd, and hit some kind of fd limits (likely 1024), and causing the hang.

if you write your own appsink and free the samples at the end, it should work

avafinger commented 1 year ago

mpp bufs holding too much fd, and hit some kind of fd limits (likely 1024)

I think it's the case, can't get fd after a while.

your own appsink and free the samples at the end

I am trying to figure out what you mean by free the samples at the end, i am kind of new to gstreamer, care to show one example?

JeffyCN commented 1 year ago

basically is to get a sample, extract the buffer, unref the sample, use the buffer, unref the buffer:

https://github.com/dkorobkov/gstreamer-appsrc-appsink-example/blob/master/JpegGstEncoder_TegraTX1.cpp#L253

https://github.com/dkorobkov/gstreamer-appsrc-appsink-example/blob/master/JpegGstEncoder_TegraTX1.cpp#L272

avafinger commented 1 year ago

@JeffyCN

Ahh, it is being unref, even so, it runs out of fd. Taken from kmscube:

Rendered 121 frames in 2.016483 sec (60.005456 fps)
Rendered 241 frames in 4.016530 sec (60.002046 fps)
Rendered 362 frames in 6.032985 sec (60.003464 fps)
Rendered 483 frames in 8.049389 sec (60.004556 fps)
Rendered 603 frames in 10.049433 sec (60.003387 fps)
Rendered 723 frames in 12.049562 sec (60.002183 fps)
Rendered 844 frames in 14.065901 sec (60.003264 fps)
Rendered 964 frames in 16.066036 sec (60.002357 fps)
could not obtain DMABUF FD: frame: 972
----- video_deinit. ----

Every last sample is being unref:

static void
set_last_frame(struct decoder *dec, EGLImage frame, GstSample *samp)
{
    if (dec->last_frame)
        dec->egl->eglDestroyImageKHR(dec->egl->display, dec->last_frame);
    dec->last_frame = frame;
    if (dec->last_samp)
        gst_sample_unref(dec->last_samp);
    dec->last_samp = samp;
}

JeffyCN commented 1 year ago

not just samples, buf needs to be unref too, you can add logs in gstmppallocator.c's buf destroy callback to confirm. and if you dup the dmafd or import it to external texture, you need to release them too.

anyway, since other sinks work well, it's unlikely an mppdec plugin issue.

JeffyCN commented 1 year ago

and if you have a simple demo, maybe i can help to check it in next week

avafinger commented 1 year ago

I don't have a simple demo, but if you have the time i would really appreciate if you could have a look. This is really an interesting example of decoding with mpp that gives a big picture for those who are going to use and learn gstreamer. gst-decoder.c is the code and changed a bit to be more stable than the original code. meson setup build cd build/ ninja ./kmscube -V /apps/Videos/tos-4096x1720-tiles.mkv

kmscube-new.tar.gz

avafinger commented 1 year ago

Hi @JeffyCN Did you have the opportunity to have a look?

JeffyCN commented 1 year ago

sorry, forget about that.

from a quick check, the leak fd is drm fd, in drm-common.c's gbm_device_get_fd, you need to close it.

JeffyCN commented 1 year ago

well, it looks like the mali is returning a dupped fd here, but mesa3d not. so it should be a bug in mali DDK.

guessing you may need to stop using this api(get the fd somewhere else), or add a special case for mali to close it.

JeffyCN / rockchip_mirrors

gstreamer hangs after a while playing hevc 10bit and av1 10bit #29