search

RidgeRun / gst-interpipe

GStreamer plug-in for interpipeline communication
Other
140 stars 63 forks source link

Failed caps renegotiation when using NVIDIA nvv4l2decoder with interpipes #135

Open rrcarlosrodriguez opened 1 year ago

rrcarlosrodriguez commented 1 year ago

PC Test

  1. Start GstD GST_DEBUG=2,*perf*:6,*EVENT*:9 gstd
  2. Create a pipeline with SW encoder, SW decoder and interpipes, at a framerate of 30fps: gstd-client pipeline_create p0 videotestsrc is-live=true ! "video/x-raw,width=640,height=480,framerate=30/1" ! videorate name=rate drop-only=true ! perf ! x264enc ! avdec_h264 ! interpipesink name=frames sync=false forward-events=true drop=false interpipesrc allow-renegotiation=true listen-to=frames format=time ! perf ! fakesink sync=false
  3. Play the pipeline gstd-client pipeline_play p0
  4. Change framerate in the videorate to 10fps: gstd-client element_set p0 rate max-rate 10

    Result

    And the behavior is as expected, there is a new caps event triggered, the renegotiation happens, and the framerate changes from 30 to 10 in the perf element, which means that everything went well.

TX2 Test

  1. Start GstD GST_DEBUG=2,*perf*:6,*EVENT*:9 gstd
  2. Create a pipeline with NVIDIA encoder, NVIDIA decoder and interpipes, at a framerate of 30fps: gstd-client pipeline_create p0 videotestsrc is-live=true ! "video/x-raw,width=640,height=480,framerate=30/1" ! videorate name=rate drop-only=true ! nvvidconv ! nvv4l2h264enc ! nvv4l2decoder ! nvvidconv ! interpipesink name=frames sync=false forward-events=true drop=false interpipesrc allow-renegotiation=true listen-to=frames format=time ! perf ! fakesink sync=false
  3. Play the pipeline gstd-client pipeline_play p0
  4. Change framerate in the videorate to 10fps: gstd-client element_set p0 rate max-rate 10

    Result

    The pipeline fails to renegotiate and it gets stuck, no more buffers flow through interpipes, because the interpipesrc is disconnected from the interpipesink.

Analysis

Based on the 2 results, it looks like the problem is not entirely caused by interpipes, but more related to a combination between interpipes and the NVIDIA decoder when renegotiation caps. Maybe the decoder is not handling properly the caps renegotiation when asked, as the SW decoder does, but that is just a theory.

hungxiu1995 commented 1 year ago

Hi, any update on this issue?

dzhang28 commented 1 year ago

I met this issue recently. It seems that once caps is negotiated, the interpipesink->caps_negotiated would be fixed. So if upstream triggers caps renegotiation with different framerate, the caps query would be failed. It's because the new caps filter passed to gst_inter_pipe_sink_get_caps would carry new framerate. But the intersection of listeners would lead to same caps result as interpipesink->caps_negotiated with old framerate. So no intersection can be made between the old interpipesink->caps_negotiated and the given filter. Then it would cause the listener being removed and no further buffer could be pushed to the listener since then.

It could be solved by https://github.com/RidgeRun/gst-interpipe/compare/master...dzhang28:gst-interpipe:hotfix/allow-caps-renegotiation-in-interpipesink But I didn't test it enough yet.