WebM bytestream format is too restrictive around "random access points"

[wolenetz]

The current text: "A SimpleBlock element with its Keyframe flag set signals the location of a random access point for that track. Media segments containing multiple tracks are only considered a random access point if the first SimpleBlock for each track has its Keyframe flag set. The order of the multiplexed blocks must conform to the WebM Muxer Guidelines."

This has multiple overly-restrictive pieces that could be relaxed, given the robustness of the coded frame processing algorithm. I suggest: 1) Replacing the first sentence with something like "A random access point for a track is signalled by a SimpleBlock element with its Keyframe flag set for that track, or a BlockGroup element having no ReferenceBlock elements for that track." 2) Drop the second sentence; use (1) and the coded frame processing algorithm. Keep the third sentence as-is.

Do these changes relax that bytestream spec too much to achieve reliable interoperability?

[jdsmith3000]

Separate from whether the coded frame processing algorithm can accommodate a relaxed spec here, why wouldn't we keep the content tightly defined? What's the advantage of relaxing the spec?

[wolenetz]

@jdsmith3000 Regarding relaxation (1-2), two parts: Content in the wild doesn't necessarily begin WebM clusters with keyframes. CFP already drops non-keyframes encountered for a track when that track's track buffer's need random access flag is true. This allows mid-segment location of first keyframe to satisfy the random access point condition for that track. More imporantly, though, WebM content is increasingly desired to be constructed out of BlockGroup blocks instead of SimpleBlock blocks, since BlockGroup blocks explicitly specify block duration. Without the clarification "or a BlockGroup element having no ReferenceBlock elements for that track", there is no MSE-defined clear way for a BlockGroup block to identify that it is a keyframe (or a random access point).

Notably, the MSE ISO-BMFF bytestream spec already defines random access points specific to individual streams (SAP types 1 or 2), not something like "a beginning of a media segment that also begins all tracks with keyframes" like the currently over-restrictive WebM bytestream spec requires.

Since Chromium previously required (for all bytestreams) every track in a media segment to begin with a keyframe and that was (of course) non-compliant/too-strict versus at least the ISO-BMFF spec (and was causing all sorts of content provider and MSE API user issues), I have direct experience with the problems that can arise from having too strict a requirement on random access points like MSE has. Chromium already employs the relaxed version proposed in this bug, too, to preclude similar MSE API user issues that we had previously seen when we were overly-strictly requiring all ISO-BMFF media segments' tracks' to begin with keyframes.

If you agree that these WebM relaxations make sense, I'll put together the PR shortly.

[wolenetz]

Such PR will also include addition of @jdsmith3000 and myself as editors of the MSE webm bytestream spec. @mwatson2 speak up if you also want to be included as co-editor on that spec.

[wolenetz]

@acolwell FYI - would you be ok with my changing of the editorship of the MSE WebM bytestream spec from: Aaron Colwell, Google Inc. to Matthew Wolenetz, Google Inc. Jerry Smith, Microsoft Corporation (and potentially Mark Watson, Netflix Inc.) Aaron Colwell (until April 2015), Google Inc.

like we did for the main MSE spec?

[acolwell]

@wolenetz LGTM for this and the other bytestream specs if/when necessary.

[wolenetz]

PR (#96) is ready for review.

[wolenetz]

@jdsmith3000 please review #96 ASAP