Spec is too rigid on requiring initial HAVE_CURRENT_DATA transition occur synchronously within coded frame processing (possibly ditto for HAVE_METADATA and init segment received processing)

[wolenetz]

Spec is too rigid on requiring initial HAVE_CURRENT_DATA transition occur synchronous during coded frame processing.

In practice, implementations (like Chrome and IIUC, Firefox) separate HTMLMediaElement.readyState transition from HAVE_METADATA to >= HAVE_METADATA based on being able to decode some or all coded frames to reach currentTime before transitioning the readyState. Further, such decode-based-transition is commonly done after firing 'updateend' on the corresponding SourceBuffer. This enables web apps to continue to append more media data to the SourceBuffer without awaiting the implementation's decode pipeline figuring out that it can actually play something at it's currentTime.

Further, Chrome has some HTMLMediaElement decoder resource scheduling code that suspends decodes under-the-hood in some cases, delaying initial reaching of HAVE_CURRENT_DATA, to optimize scarce resource utilization (decoder, for example). Likewise, not blocking 'updateend' for such MSE-extended elements allows apps to append more media.

@jyavenard, I suspect relaxing the coded frame processing step to reflect what implementations have chosen to do, is probably the best way forward. What do you think?

[wolenetz]

Note, this is part of why Chrome is failing the wpt/media-source/mediasource-append-buffer.html's 'Test appendBuffer events order.' case. See https://crbug.com/641121 for more info (there's a similar problem with initial HAVE_METADATA transition, but that may be more reasonable to retain in spec; though it also can delay 'updateend' and further media appending by app in at least Chrome and IIUC, Firefox.

[jyavenard]

Indeed, this is one area of the spec we do not follow. It wouldn't be possible for us to implement it for some content anyway.

Like say you append a media segment made of a single opus frame. Technically there's a buffered range as far as the track buffer is concerned. But decoding it would result in no output by the decoder (due to pre-roll). With nothing to play, readyState would still be at HAVE_METADATA

[jyavenard]

We have implemented the requirement that HAVE_METADATA transition occurs during the Initialization Segment Received step however.

Gecko delays the firing of updateend, until the processing of the init segment has trickled all the way back to the media element. Even if the work/transition doesn't actually happen when the spec says it should be. Observable result is identical

[wolenetz]

See also #275