search

w3c / media-source

Media Source Extensions
https://w3c.github.io/media-source/
Other
268 stars 58 forks source link

Will MSE Support audio/ogg (audio/opus) Byte Streams? #245

Open anthumchris opened 4 years ago

anthumchris commented 4 years ago

Will Media Source Extensions support byte streams for Ogg Opus files? <audio> elements can currently play audio/ogg Opus files but the MSE Byte Stream Registry does not specify support for MSE, which feels inconsistent.

The official Opus encoder and file tools (opusinfo and opusenc and opusdec) only support files with Ogg encapsulation. Looks like Chrome/Firefox are supporting opus in other container formats though:

chrome

Screen Shot 2019-12-02 at 11 03 12 PM

firefox

Screen Shot 2019-12-02 at 11 07 05 PM
guest271314 commented 4 years ago

Interestingly Chromium does not play "audio/webm;codecs=opus" using MediaSource and Firefox plays a file actually encoded as Vorbis when "audio/webm;codecs=opus" is passed to addSourceBuffer() https://github.com/web-platform-tests/wpt/issues/21240.

guest271314 commented 4 years ago

Tried using ReadableStream and AudioWorklet yesterday which plays when ~100kb of data has been received by fetch(). Am able to get the audio to play after roughly 42K bytes. However, requires re-reading the entire stored array values as ArrayBuffer subsequently for decodeAudioData().

Can you not write Opus audio to a WebM container in the meantime?

Or, splice the single file into N size files, each with a header so that decodeAudioData() can read the rest of the audio data?

anthumchris commented 4 years ago

Yes, we could use those workarounds, but the main goal would be to offer implicit browser support of audio/ogg without workarounds.

wolenetz commented 4 years ago

Bytestream spec PR is welcome.