search

xiph / opus-tools

A set of tools to encode, inspect, and decode audio in the Opus format.
https://opus-codec.org/
Other
224 stars 79 forks source link

WebM Container Support for Opus Tools #49

Open anthumchris opened 4 years ago

anthumchris commented 4 years ago

How feasible would it be to add official tooling support for WebM Containers? Like Ogg, WebM seems to be a preferred, royalty-free web standard moving forward. Currently, Ogg containers are not supported by Media Source Extensions (MSE), but MSE does support WebM (see https://github.com/w3c/media-source/issues/245). I am currently using third-party tools like ffmpeg to create WebM Opus audio files for use with MSE.

Use Case

Based on my research & experimentation, MSE currently provides the lowest possible latency for Opus playback in browsers with slow connections, allowing audio to be played as soon as the first audio bytes are downloaded. Additionally, MSE offers native platform Opus decoding without needing to use WebAssembly and the Web Audio API to decode and play PCM audio. Would be great to use official Opus tools to achieve this.

Without MSE, native browser playback requires up to 100k to be downloaded (HTTP range requests for start (metadata) and last (duration calculation) bytes). This is problematic for slow internet connections capable of sustainably streaming an Opus file. For example, a 64 bitrate Opus file over a 72 kbps connection yields a 12s playback latency in Chrome and 5s in Firefox:

https://fetch-stream-audio.anthum.com/72kbps/opus/house--64kbs.opus?cacheBust=1

Screen Shot 2019-12-03 at 10 46 03 AM

(more throttled endpoints)

anthumchris commented 4 years ago

I haven't been able to determine what MSE bytestreams Safari supports. I probed with:

console.log([
  MediaSource.isTypeSupported('audio/webm; codecs="opus"'),
  MediaSource.isTypeSupported('audio/mp4; codecs="opus"'),
  MediaSource.isTypeSupported('audio/mp2t; codecs="opus"'),
  MediaSource.isTypeSupported('audio/mpeg; codecs="opus"'),
  MediaSource.isTypeSupported('audio/aac; codecs="opus"')
])
guest271314 commented 2 years ago

@AnthumChris

Currently, Ogg containers are not supported by Media Source Extensions (MSE)

FYI: MSE on Chromium 105 now supports passing ('opus') EncodedAudioChunk from WebCodecs https://plnkr.co/edit/cAbZqVXfzZMFOaxL?preview.

guest271314 commented 2 years ago

@AnthumChris

See https://github.com/guest271314/WebCodecsOpusRecorder. From my testing the WebCodecs configuration and Opus packets in a single file has less total size than Opus in WebM produced by Chromium MediaRecoder implementation.

vadimkantorov commented 1 year ago

it would already be useful even if it's just supported in opusdec for demuxing+decoding

vadimkantorov commented 10 months ago

Btw it appears that mkv + opus can significantly save space on omitting silence: https://video.stackexchange.com/questions/37055/possible-to-force-ffmpeg-to-omit-silence-in-audio-files-and-produce-discontinuou/37056?noredirect=1#comment47208_37056

if opusenc can support output webm streams + silence suppression, it would be very practical and not force ffmpeg usage and be faster