w3c / media-source

Media Source Extensions
https://w3c.github.io/media-source/
Other
267 stars 59 forks source link

Standardize expectations around endOfStream followed by later re-opening of MediaSource #299

Open wolenetz opened 2 years ago

wolenetz commented 2 years ago

endOfStream() is a useful mechanism for apps to let the HTMLMediaElement know when playback has reached the end of the media.

However, there are complications involved with the MSE spec enabling apps to call endOfStream(), and subsequently re-open the MediaSource (various operations can re-open an 'ended' MediaSource, such as doing an appendBuffer() to one of its SourceBuffers).

Primarily, the complications occur due to varying media decoding and rendering pipeline depths in implementations:

If the endOfStream() occurs at a point when all media for a particular track buffer has been fed to the decoder already, and the implementation, following the MSE spec, executes "Notify the media element that it now has all of the media data", then what should happen later upon a re-opening of the MediaSource?

Chromium, upon such a re-opening, does not reactivate a track's decoder/renderer pipeline to potentially resume from where it was previously if it had previously reached the end. Since the media element already had all of its media data, and had potentially finished rendering it, is not reactivating the decoders/renderer pipeline for previously finished media correct upon MediaSource re-opening? (For instance, if the HTMLMediaElement had completely reached 'ended', but then the MediaSource were re-opened with more media appended to the end, Chromium doesn't automatically resume playback).

This bug tracks discussing if this behavior is correct. It can be a source of unexpected behavior, especially if implementations differ on their interpretation of what to do upon re-opening the MediaSource (Safari, IIRC may do this differently.)