Is HTMLVideoElement.canplay event expected to be dispatched following MediaSource.appendBuffer()?

[guest271314]

According to the specification

Event name canplay

Interface Event

Fired when... The user agent can resume playback of the media data, but estimates that if playback were to be started now, the media resource could not be rendered at the current playback rate up to its end without having to stop for further buffering of content.

Preconditions readyState newly increased to HAVE_FUTURE_DATA or greater.

HAVE_FUTURE_DATA (numeric value 3)

Data for the immediate current playback position is available, as well as enough data for the user agent to advance the current playback position in the direction of playback at least a little without immediately reverting to the HAVE_METADATA state, and the text tracks are ready. For example, in video this corresponds to the user agent having data for at least the current frame and the next frame when the current playback position is at the instant in time between the two frames, or to the user agent having the video data for the current frame and audio data to keep playing at least a little when the current playback position is in the middle of a frame. The user agent cannot be in this state if playback has ended, as the current playback position can never advance in this case.

When an ArrayBuffer is appended to a MediaSource instance using .appendBuffer() canplay is fired at Chromium 60, though not at Firefox 55.

Is canplay expected to be fired at Firefox when an ArrayBuffer is appended to MediaSource instance?

When autoplay attribute is set to true, why is video.paused true at Firefox 55, though not at Chromium 60, where duration of the media as array buffer is approximately one second?

Why are there inconsistent implementations at Chromium and Firefox?

Not sure if the issue is a Blink or Gecko bug.

Relevant code

HTML

<video preload="auto" autoplay="true" width="320" height="280" controls="true"></video>

JavaScript

    // ..
    const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
    sourceBuffer.appendWindowStart = 0;
    sourceBuffer.appendWindowEnd = mediaDuration;
    sourceBuffer.onupdateend = e => {
      sourceBuffer.onupdateend = null;
      console.log(mediaDuration, mediaSource.duration);
      video.currentTime = 0;
      // chromium 60 renders expected result
      // firefox 55 does not reach `canplay` event
      // chromium 60: `1 false false 0`
      // firefox 55 `1 true false 0`
      // firefox issue here
      console.log(video.readyState, video.paused, video.ended, video.currentTime);
      video.oncanplay = async(e) => {
          video.oncanplay = null;
          console.log(e);
          console.log("media source playing", video.readyState);
          // firefox issue here, we need playback to continue
          const play = await video.play();
          // ..
      }
      sourceBuffer.appendBuffer(mediaBuffer);
     // ..

[guest271314]

Related https://github.com/whatwg/html/issues/3002

[jyavenard]

Please retry with Firefox beta 56, there was a bug in 55 preventing readyState to progress if your data didn't start at exactly zero.

How much data are you adding to the source buffer? What is the buffered range after appendBuffer has completer?

[guest271314]

@jyavenard Will try to install and try at Firefox beta 56. The candidate by default at package manager is still 55.0.2 according to apt-cache.

The data amount is roughly one second of recorded media. Just tried to reproduce issue again and error was logged at console

Media resource https://upload.wikimedia.org/wikipedia/commons/2/22/Volcano_Lava_Sample.webm#t=10,11 could not be decoded. OfH5OfNChz51liz9 Media resource https://upload.wikimedia.org/wikipedia/commons/2/22/Volcano_Lava_Sample.webm#t=10,11 could not be decoded, error: Error Code: NS_ERROR_DOM_MEDIA_DEMUXER_ERR (0x806e000c) OfH5OfNChz51liz9

Where can we source valid .mp4, .webm and other files types for testing MediaSource online?

[jyavenard]

https://upload.wikimedia.org/wikipedia/commons/2/22/Volcano_Lava_Sample.webm#t=10,11 is not a valid URL

[guest271314]

@jyavenard

https://upload.wikimedia.org/wikipedia/commons/2/22/Volcano_Lava_Sample.webm#t=10,11 is not a valid URL

What do you mean?

[guest271314]

@jyavenard See Media Fragments URI 1.0 (basic)

[guest271314]

@jyavenard Just tried with URL "https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=10,11"

and 1.00136 1.125 96434 0.056 0.986

were logged for

console.log(mediaDuration, mediaSource.duration);
console.log(mediaBuffer.byteLength, sourceBuffer.buffered.start(0), sourceBuffer.buffered.end(0));

The issue appears to have been the invalid .webm file. Was able to get expected video result at Firefox 55, save for needing to use AudioContext.createMediaElementSource(), AudioContext.createMediaStreamDestination(), HTMLCanvasElement.captureStream() for MediaStream instead of HTMLMediaElement.captureStream(), which does not playback audio due to cross-origin request, though that is another issue.

[guest271314]

@jyavenard fwiw, the code which have composed so far relevant to the current issue https://github.com/guest271314/recordMediaFragments. Firefox implementation still has several issues.

[jyavenard]

So your segment doesn't start at 0. There is an issue in Firefox 55 where autoplay attribute will not work under those conditions. You must manually call play() this is fixed in 56.

[guest271314]

@jyavenard Installed Firefox 57, removed autoplay="true" at <video> element, still getting same result. canplay is dispatched three times, though not for each media fragment appended as a buffer. The procedure stops at some point between events between waiting and canplay events being dispatched.

[guest271314]

@jyavenard Will re-read the specification and attempt to make adjustments