New Playback Requirement: Source Buffer Re-Initialization

[haudiobe]

8.15 Source Buffer Re-Initialization This test provides a stimulus in case that a source buffer needs to be re-established, for example because of a codec change or a codec parameter change. The first version of this specification does not define a test for this.

[haudiobe]

(DPCTF 19/11/06)

• Nothing in the content specification except specific splice conditioning.
• MSE adds the ability to do codec changes. This is in v2 of MSE.

From DCPTF Spec perspective you need:

• Capability discovery: Do you support switching across codecs or codec profiles?
• If supported, then we need to have Stimulus Add one CMAF Track, change CMAF Header, continue with next CMAF Track Observation Expect seamless playback, but need more definition. Do we permit non-seamlessness to some extent? ** We need test cases and test material

[haudiobe]

2020/07/08:

New API is available: https://developer.mozilla.org/en-US/docs/Web/API/SourceBuffer/changeType

if Sequence:

• append media old type
• try if changeType() is available
  • if yes, do changeType()
  • if no, do teardown and re-establish source buffer buffer
• append Header of new type
• append Media of new type

This should be added to the tests.

Observation Expectation:

• codec switch happens
• seamless?
• at least working with a glitch small enough tbd

[dsilhavy]

What @haudiobe describes reflects what we are doing in dash.js today:

• We check if changeType is supported or if the media codecs are compatible. For the latter we check
  • if the codecs are an exact match including level and profile (for instance transition from "avc1.64001e" to "avc1.64001e")
  • or if they partially match and have the same mime type. For avc we expect avc1 and avc3 to be compatible and not require a reinit of the SourceBuffer.
• In all other cased we teardown the MediaSource and reattach the SourceBuffers. For instance transition from avc to hevc and no changeTypeavailable.

Teardown will result in a small glitch especially because upcoming segments are not prebuffered and are downloaded once the MSE has been reset.

[jpiesing]

It's not completely clear to me whether this is one test or two or indeed three? I think there clearly is a test for changeType. Care might be needed about the combinations of codecs to be tested here. Obviously this test would only be run on implementations supporting changeType. Is a test of teardown needed for implementations not supporting changeType? Is a test needed of sending an initialization segment even when the codecs are the same just to check that the implementation doesn't re-initialize when it doesn't need to?

[dsilhavy]

Maybe it would be good to do two tests separate tests here:

• changeType() test. If not supported test might be ignored or counts as failed
• Teardown and reinit the MSE + Sourcebuffers before appending the new CMAF header + CMAF segment. There will always be a small glitch as mentioned by Thomas. We could measure the time between stall -> reinit -> playing state in a simple test case.

[joeyparrish]

MediaCapabilities will indicate whether or not changeType is supported for the chosen media pipeline. For example, changeType might be implemented by the UA, but the platform's hardware decoding pipeline might not be able to change codecs once initialized.

[andyburras]

The changeType() test raises an interesting question. The HATF Web Media API Snapshot 2020 https://w3c.github.io/webmediaapi/ references Media Source Extensions W3C Recommendation 17 November 2016 https://www.w3.org/TR/media-source/ which I don't think specifies SourceBuffer.changeType(). Are we OK to use JavaScript methods in tests that are outside the CTA WAVE HATF spec?

[haudiobe]

part of the new tests