search

w3c / media-source

Media Source Extensions
https://w3c.github.io/media-source/
Other
268 stars 58 forks source link

Does the specification require duration to be set for all blocks in WebM compliant files? #250

Open guest271314 opened 4 years ago

guest271314 commented 4 years ago

Creating a WebM compliant file from an audio/x-opus+ogg file using mkvmerge (https://mkvtoolnix.download/doc/mkvmerge.html#d4e612)

-w, --webm | Create a WebM compliant file. This is also turned on if the output file name's extension is "webm". This mode enforces several restrictions. The only allowed codecs are VP8, VP9 video and Opus, Vorbis audio tracks. The DocType header item is changed to "webm".For chapters and tags only a subset of elements are allowed. mkvmerge(1) will automatically remove all elements not allowed by the specification.

without --enable-durations option

Write durations for all blocks. This will increase file size and does not offer any additional value for players at the moment.

results in an exception being thrown at Chromium 81 (https://bugs.chromium.org/p/chromium/issues/detail?id=1042988)

DOMException: Failed to execute 'appendBuffer' on 'SourceBuffer': The HTMLMediaElement.error attribute is not null.

When --enable-durations is used the WebM file does not throw an execption and plays back the file.

The relevant section in the specification appears to be 3.5.7 Initialization Segment Received https://w3c.github.io/media-source/#sourcebuffer-init-segment-received

  1. Update the duration attribute if it currently equals NaN:
    • If the initialization segment contains a duration:
    • Run the duration change algorithm with new duration set to the duration in the initialization segment.
    • Otherwise:
    • Run the duration change algorithm with new duration set to positive Infinity.

which does not indicate duration MUST be set for all blocks.

If there is in fact technical language in the specifcation which refers to MediaSource implementation expecting duration for all blocks in WebM compliant files can those specifc sections and language kindly be pointed out here?

wolenetz commented 4 years ago

See https://bugs.chromium.org/p/chromium/issues/detail?id=1042988#c15 Recommendation is, yes, include block durations in bytestreams muxed for use with MSE so that implementations can precisely buffer the media. Unlike traditional linear buffering, MSE allows segments to be relocated in the presentation timeline. Determinining the precise buffering overlap and adjacency requires accurate coded frame duration information. In this case, it seems to me that the comment "This will increase file size and does not offer any additional value for players at the moment." is incorrect w.r.t. MSE players. The spec references in the OP to the initialization segment logic are around the stream duration, not coded frame duration.

See also https://github.com/w3c/media-source/pull/234, where this recommendation is hoped to be merged into the MSE bytestream format spec for WebM soon.

guest271314 commented 4 years ago

Given the various implementations of WebM writers one solution which would resolve this issue and the linked PR would be to create a WebM writer that outputs WebM files with the container structure MediaSource SourceBuffer with codecs set to "<audio|video>/webm;codecs=<codec(s)>" expects per any (including the current) iteratation of https://www.w3.org/TR/mse-byte-stream-format-registry/.