Dynamic browser support

[AmitMY]

Using this library, I generate videos on the fly.

Then, I try to play the video in the browser.

In chrome desktop, it works, but on safari (desktop/mobile) or chrome mobile it doesn't. See example file: test.webm

I could not yet figure out why this is the case. I would like this library to support an isPlayable method, that given a Muxer determines if the video is playable or not. It should return either true, false, or null.

Quick mock implementation:

async function isPlayable() {
    if (!('mediaCapabilities' in navigator)) {
      return null;
     // or maybe use `canPlayType` as a fallback. or `MediaSource.isTypeSupported(mimeType)`
    }

    const videoConfig = {
      contentType: 'video/webm; codecs="vp09.00.10.08"', // replace with codec
      width: 1280,  // Replace with actual width
      height: 720,  // Replace with actual height
      bitrate: 1000000,  // Replace with actual bitrate
      framerate: 25,  // Replace with actual frame rate
      hasAlphaChannel: true, // replace with alpha
    };
   // {"powerEfficient":true,"smooth":true,"supported":true,"supportedConfiguration":{"video":{"bitrate":1000000,"contentType":"video/webm; codecs=\"vp09.00.10.08\"","framerate":25,"height":720,"width":1280},"type":"file"}}
    const result = await navigator.mediaCapabilities.decodingInfo({type: 'file', video: videoConfig});
    return result.supported; 
}

Or maybe, for example here, we specify hasAlphaChannel: true but the supportedConfiguration says no alpha is supported, we might be able to makePlayable by using the specified configuration

[Vanilagy]

The video plays for me in Safari (17.2):

Older versions of Safari didn't support WebM, and I think mobile Safari is still flakey:

It should play on Chrome mobile. If you tested Chrome mobile on an iOS device, you didn't use Chromium, you used WebKit.

---

As the goal of this library is only to create WebM videos, I say an isPlayable is beyond its scope of responsibility. Compatibility has a lot to do with the codecs at play, which the muxer has no idea about. That's what you configure when you set up the VideoEncoder. Your isPlayable method looks good, I say just use that.

[Vanilagy]

By the way, that sign translator is really awesome!!

[AmitMY]

Thank you for the quick response and information :)

I have now created something to use the right muxer given browser support - https://github.com/sign/translate/blob/master/src/app/pages/translate/pose-viewers/playable-video-encoder.ts

And you are right, safari does support this file, but for some reason, not before I download it. Very strange, I am not sure how to make this work (in Chrome everything works fine). Here's a reproduction of it not working, but supporting a file:

https://github.com/Vanilagy/webm-muxer/assets/5757359/8160af8c-637c-43d5-849c-8ec9aa376323

[Vanilagy]

Very sus, does this happen in all browsers, or just Safari? Is the Blob maybe incomplete or something when you first load it into the video, or slightly different when you then download it? Clearly, the video is valid and should play back just fine.

[AmitMY]

A bug in safari. Here's my fix https://github.com/sign/translate/commit/905b8cf3dec42bd62764ae3238664bf6c00c4727#diff-4cca635d509f0a6c08cc233cc13337620521cda035bd4a935a83832775034fb8R93-R105

My apologies for opening this unrelated-to-this-library issue, I did not expect it to be a browser issue :)

[Vanilagy]

Browsers are still full of subtle media bugs, no problem!