Closed avafinger closed 1 year ago
Additional info:
0:00:19.394285934 15753 0x7fb4067460 ERROR mppdec gstmppdec.c:1012:gst_mpp_dec_handle_frame:<mppvideodec0> failed to send packet
0:00:19.395494857 15753 0x7fb4067460 WARN mppdec gstmppdec.c:1016:gst_mpp_dec_handle_frame:<mppvideodec0> can't handle this frame
0:00:19.396545702 15753 0x5567d58de0 WARN matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: Internal data stream error.
0:00:19.397677044 15753 0x5567d58de0 WARN matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: streaming stopped, reason error (-5)
0:00:19.398770179 15753 0x5567d58de0 INFO GST_ERROR_SYSTEM gstelement.c:2271:gst_element_message_full_with_details:<matroskademux0> posting message: Internal data stream error.
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
0:00:19.405222826 15753 0x5567d58de0 INFO GST_ERROR_SYSTEM gstelement.c:2298:gst_element_message_full_with_details:<matroskademux0> posted error message: Internal data stream error.
Execution ended after 0:00:19.301138668
Setting pipeline to NULL ...
0:00:19.407927956 15753 0x5567b80e00 INFO GST_STATES gstbin
Not sure if it is related but i have this patch applied: https://raw.githubusercontent.com/JeffyCN/rockchip_mirrors/buildroot/package/gstreamer1/gst1-plugins-good/0009-matroska-demux-parse-Disable-QoS-lace-skipping-by-de.patch
rk3588, gstreamer 1.20.3 with all patches applied. FFmpeg play these files just fine, so mpp is working.
looks like the mpp's put packet hang.
maybe try to increase the input timeout: -#define MPP_INPUT_TIMEOUT_MS 2000 / Block timeout for MPP input queue / +#define MPP_INPUT_TIMEOUT_MS 2000000 / Block timeout for MPP input queue /
and use gdb to debug it.
I built your latest gstreamer-rockchip, still has the same error:
0:00:19.446961334 26242 0x7f88066460 ERROR mppdec gstmppdec.c:1036:gst_mpp_dec_handle_frame:<mppvideodec0> failed to send packet
0:00:19.447033081 26242 0x7f88066460 WARN mppdec gstmppdec.c:1040:gst_mpp_dec_handle_frame:<mppvideodec0> can't handle this frame
0:00:19.447211284 26242 0x7f88066460 INFO task gsttask.c:368:gst_task_func:<multiqueue0:src_0> Task going to paused
0:00:19.447313365 26242 0x55923d7de0 WARN matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: Internal data stream error.
0:00:19.447345739 26242 0x55923d7de0 WARN matroskademux matroska-demux.c:6076:gst_matroska_demux_loop:<matroskademux0> error: streaming stopped, reason error (-5)
I increased the values of the timeout_ms (5000) and interval_ms (10), same error.
Comparing this to FFmpeg, in which there is no timeout on decode_put_packet, it assumes "Decoder buffer full" and return EAGAIN and then you comment: "some streams might need more packets to start returning frame..." and get next frame.
I don't see this in gstreamer-rockchip, isn't that the case?
ffmpeg and gst are different, i don't think that would be related.
please increase the timeout to confirm if it hang in the put packet, so that we could use gdb to debug it
please increase the timeout to confirm if it hang in the put packet, so that we could use gdb to debug it
Yes, it hangs. Unfortunately, i don't know how to debug gstreamer with gdb (i know gdb basics). Is there a way to build gstreamer-rockchip with debugging (without stripping and optimizations)? I might debug it using Geany (which uses gdb, :) ) .
ffmpeg and gst are different
Yeah, but a see a pattern:
gstreamer:
mpp_packet_set_pts()
decode_put_packet (); -> if error on timeout, return error
decode_get_frame();
ffmpeg:
ret = decode_get_frame();
return decode_put_packet ();
if (ret != MPP_OK)
return (EAGAIN)
decode_get_frame();
Redistribute latency... New clock: GstSystemClock 0:00:16.6 / 0:00:30.0 (55.2 %)
Hang at 00:16.6 every time i run.
gst-launch-1.0 filesrc location=jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! appsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
klass->send_mpp_packet() rc: 0 - timed out with error
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst/matroska/matroska-demux.c(6076): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason error (-5)
Execution ended after 0:01:12.808293867
Setting pipeline to NULL ...
Freeing pipeline ...
file: wget http://www.larmoire.info/jellyfish/media/jellyfish-10-mbps-hd-hevc-10bit.mkv
Additional info, this can decode and display on X11 just fine:
DISPLAY=:0.0 gst-launch-1.0 filesrc location=~/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! xvimagesink
Decode but can't display (black screen)
DISPLAY=:0.0 gst-launch-1.0 filesrc location=~/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! rkximagesink
Crashed:
DISPLAY=:0.0 gst-launch-1.0 filesrc location=~/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! glimagesink
Setting pipeline to PAUSED ...
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
**
GStreamer-GL:ERROR:../gst-libs/gst/gl/gstglformat.c:234:gst_gl_format_from_video_info: code should not be reached
Bail out! GStreamer-GL:ERROR:../gst-libs/gst/gl/gstglformat.c:234:gst_gl_format_from_video_info: code should not be reached
Aborted (core dumped)
and these pipelines do not work (wayland/weston):
gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-20-mbps-hd-hevc-10bit.mkv ! matroskademux ! h265parse ! mppvideodec ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.001810498
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.004942712
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.001950491
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! kmssink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: GStreamer encountered a general resource error.
Additional debug info:
../sys/kms/gstkmssink.c(2063): gst_kms_sink_show_frame (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
drmModeSetPlane failed: Invalid argument (22)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$ gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! kmssink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: GStreamer encountered a general resource error.
Additional debug info:
../sys/kms/gstkmssink.c(2063): gst_kms_sink_show_frame (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
drmModeSetPlane failed: Invalid argument (22)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
rock@rock5b:~$
1/ if it hang in put packet, then add logs to get frame, if it hang there too, it's an mpp issue
2/ nv12 10 might need more patches in the sink, please try to get 1.20 version patches here: https://github.com/JeffyCN/rockchip_mirrors/commit/9ee48e73ec313323df20f3e69c964dfe46bb0fb4
it works well on my 3588 evb with buildroot sdk: root@rk3588:/# killall weston root@rk3588:/# gst-play-1.0 /jellyfish-10-mbps-hd-hevc-10bit.mkv --videosink=kmssink 按“k”键来显示键盘快捷键列表。 正在播放 /jellyfish-10-mbps-hd-hevc-10bit.mkvRedistribute latency... Redistribute latency... Redistribute latency... 0:00:30.0 / 0:00:30.0 到达播放列表结尾。
FYI: 1/ i'm using mpp at: https://github.com/JeffyCN/mirrors/tree/mpp-dev 2/ the gst 1.20 patches at https://github.com/JeffyCN/rockchip_mirrors/commit/9ee48e73ec313323df20f3e69c964dfe46bb0fb4 doesn't contain the position patches(glimagesink and xvimagesink), you may need to add those manually 3/ for waylandsink, you may need to set env "export GST_MPP_DEC_DMA_FEATURE=1" to make waylandsink use dmabuf 4/ for kmssink, you may need to set plane-id property to an non-cluster overlay plane for non-afbc output formats, and maybe configure the plane's zpos property with modetest tool 5/ for rkximagesink, same as kmssink
Thanks for the info,
I think i have that patches, i will revise . I am particularly interested in appsink so i can use gstreamer programmatically.
Can you please run this pipeline and report the result? gst-launch-1.0 filesrc location=jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! appsink
you may need to write a real appsink to use it, otherwise it will no free the output frames/buffers by default, check it's max-buffers and drop properties.
or please test it with other existing sink.
or please test it with other existing sink.
@JeffyCN
I already did that and I have the same problem.
will no free the output frames/buffers by default, check it's max-buffers and drop properties.
Trying to figure out how to do this.
max-buffers = 2 No idea about the drop properties, can you give some hints?
So, that means these pipelines:
gst-launch-1.0 filesrc location=jellyfish-10-mbps-hd-hevc-10bit.mkv ! decodebin ! video/x-raw,format=NV12_10LE40 ! appsink
and
gst-launch-1.0 filesrc location=/apps/Videos/jellyfish-20-mbps-hd-hevc-10bit.mkv ! matroskademux ! h265parse ! mppvideodec ! appsink
are expected to fail for the above reason?
1/ as replied before, the waylandsink glimagesink kmssink rkximagesink xvimagesink fakesink both works(some of them needs to set env or property)
2/ appsink drop=true max-buffers=1 'gst-inspect-1.0 appsink' to check the meaning of those properties, basically is to only keep 1 buffer in queue and drop the old buffers.
otherwise it will hold buffers and not to free them, for example 'gst-launch-1.0 videotestsrc ! appsink' you can see the memory usage keep increasing.
appsink drop=true max-buffers=1
Works, thank you.
guessing the increasing mpp bufs holding too much fd, and hit some kind of fd limits (likely 1024), and causing the hang.
if you write your own appsink and free the samples at the end, it should work
mpp bufs holding too much fd, and hit some kind of fd limits (likely 1024)
I think it's the case, can't get fd after a while.
your own appsink and free the samples at the end
I am trying to figure out what you mean by free the samples at the end, i am kind of new to gstreamer, care to show one example?
basically is to get a sample, extract the buffer, unref the sample, use the buffer, unref the buffer:
@JeffyCN
Ahh, it is being unref, even so, it runs out of fd. Taken from kmscube:
Rendered 121 frames in 2.016483 sec (60.005456 fps)
Rendered 241 frames in 4.016530 sec (60.002046 fps)
Rendered 362 frames in 6.032985 sec (60.003464 fps)
Rendered 483 frames in 8.049389 sec (60.004556 fps)
Rendered 603 frames in 10.049433 sec (60.003387 fps)
Rendered 723 frames in 12.049562 sec (60.002183 fps)
Rendered 844 frames in 14.065901 sec (60.003264 fps)
Rendered 964 frames in 16.066036 sec (60.002357 fps)
could not obtain DMABUF FD: frame: 972
----- video_deinit. ----
Every last sample is being unref:
static void
set_last_frame(struct decoder *dec, EGLImage frame, GstSample *samp)
{
if (dec->last_frame)
dec->egl->eglDestroyImageKHR(dec->egl->display, dec->last_frame);
dec->last_frame = frame;
if (dec->last_samp)
gst_sample_unref(dec->last_samp);
dec->last_samp = samp;
}
not just samples, buf needs to be unref too, you can add logs in gstmppallocator.c's buf destroy callback to confirm. and if you dup the dmafd or import it to external texture, you need to release them too.
anyway, since other sinks work well, it's unlikely an mppdec plugin issue.
and if you have a simple demo, maybe i can help to check it in next week
I don't have a simple demo, but if you have the time i would really appreciate if you could have a look. This is really an interesting example of decoding with mpp that gives a big picture for those who are going to use and learn gstreamer. gst-decoder.c is the code and changed a bit to be more stable than the original code. meson setup build cd build/ ninja ./kmscube -V /apps/Videos/tos-4096x1720-tiles.mkv
Hi @JeffyCN Did you have the opportunity to have a look?
sorry, forget about that.
from a quick check, the leak fd is drm fd, in drm-common.c's gbm_device_get_fd, you need to close it.
well, it looks like the mali is returning a dupped fd here, but mesa3d not. so it should be a bug in mali DDK.
guessing you may need to stop using this api(get the fd somewhere else), or add a special case for mali to close it.
Hi @JeffyCN ,
I am trying to decode hevc 10-bit and av1 10-bit, it works for a while and then gives an internal error, exactly at the same offset anytime i play the file.
You can reproduce the error with:
HEVC 10bit:
verbose:
AV1 10bit: