WebPlatformForEmbedded / WPEWebKit

WPE WebKit port (downstream)
213 stars 136 forks source link

wpe-2.38 MSE Gstreamer: Push STREAM_COLLECTION event after flushing #1197

Closed asurdej-comcast closed 11 months ago

asurdej-comcast commented 11 months ago

Flushing the playback pipeline when very first buffers are being pushed downstream causes STREAM_COLLECTION event to be dropped before reaching decodebin3. As a result playback never starts after flush (buffer push return not-linked). This happens on initial seek sometimes, when it is triggered at the same time that first buffers push.

STREAM_COLLECTION is dropped by GstBaseParse element specificaly that is caching events and sending them with first frame. Unfortunately flushing will drop all pending events if first frame is not ready yet.

asurdej-comcast commented 11 months ago

To be more specific this STREAM_COLLECTION is cached inside GstBaseParse (h264/265parser and audio parse) in

static gboolean
gst_base_parse_sink_event_default (GstBaseParse * parse, GstEvent * event)
{
...
  if (event) {
    if (!GST_EVENT_IS_SERIALIZED (event) || forward_immediate) {
      ret = gst_pad_push_event (parse->srcpad, event);
    } else {
      parse->priv->pending_events =
          g_list_prepend (parse->priv->pending_events, event);
      ret = TRUE;
    }
  }

pending_events array is cleared on flushing and all events are dropped. I also prepared a change for GstBaseParse like:

diff --git a/libs/gst/base/gstbaseparse.c b/libs/gst/base/gstbaseparse.c
index 185e811..87777a5 100644
--- a/libs/gst/base/gstbaseparse.c
+++ b/libs/gst/base/gstbaseparse.c
@@ -1532,6 +1532,12 @@ gst_base_parse_sink_event_default (GstBaseParse * parse, GstEvent * event)
       parse->priv->tags_changed = TRUE;
       break;
     }
+    case GST_EVENT_STREAM_COLLECTION:
+    {
+      if (parse->priv->pad_mode != GST_PAD_MODE_PULL)
+        forward_immediate = TRUE;
+      break;
+    }
     default:
       break;
   }

but not sure if this is expected for all use-cases.

In latest GST there were couple of changes around this and STREAM_COLLECTION seems to be handled in gstdecodebin sink pad directly so it happens before parser element.

asurdej-comcast commented 11 months ago

I've also tried to reset hasPushedStreamCollectionEvent only if first buffer push returned FLUSHING, which works fine for h264/5 videos but ac3parse audio seems to consume couple of buffers before producing a frame (and pushing events forward) so we have no way to be sure that STREAM_COLLECTION really reached its destination

eocanha commented 11 months ago

Patch submitted upstream for review as https://bugs.webkit.org/show_bug.cgi?id=262193 and https://github.com/WebKit/WebKit/pull/18285.

eocanha commented 11 months ago

Commit landed upstream as https://github.com/WebKit/WebKit/commit/4c9c583641452ecb3807a9fa58103629431710cf and ported to wpe-2.38 as https://github.com/WebPlatformForEmbedded/WPEWebKit/commit/3b2979e985b6546aea15d828593d4a2b5b4e37b7