AC3 and DTS sound support

[piotrkustal]

desktop web 0.3.3

win10-64

trying to play mkv video with dts sound

to hear sound

no sound

[rom1504]

https://github.com/feross/webtorrent/issues/405

[feross]

@rom1504 This is actually a different issue than feross/webtorrent#405. WebTorrent Deskop doesn't use webtorrent's file.appendTo() functionality since it can just start an http server with torrent.createServer() and point the <video> tag to the http url. This ends up supporting more file formats.

[rom1504]

Ah yeah I see. So I guess this is more of a chrome (non-)support problem ?

[rom1504]

https://bugs.chromium.org/p/chromium/issues/detail?id=243861

[rom1504]

Won't fix, because AC3 is owned by Dolby and to support it would mean that royalties would have to be paid to Dolby to use it, which goes against an open-source software like Chromium.

so I think the only way for this to work is to have some package convert dts

[dcposch]

First, the bad news: WebChimera doesn't look like it will work for now. It is flaky and just crashes the renderer process about 50% of the time I try to play a video.

Good news: we might not need WebChimera. I figured out how to recompile electron and libchromiumcontent with different flags. If we compile in Chromecast mode (ie, like Chrome for Chromecast), it should support AC3 and DTS in the standard <video> and <audio> tags.

More bad news: I can't even reproduce the original problem, where a video plays but the audio doesn't.

This version of Tears of Steel is an MP4 with AC3-encoded audio, so it should play video-only with muted audio, but for me that doesn't play at all, not even the video.

magnet:?xt=urn:btih:16c57728d7e5143d69a4f15d22b8894646165c51&dn=tears-of-steel-ac3.mp4&tr=udp%3A%2F%2Fexodus.desync.com%3A6969&tr=wss%3A%2F%2Ftracker.webtorrent.io

Does anyone have a video where only the audio track is broken? Must be copyleft / freely shareable.

[grunjol]

Kodi has samples for testing. The non-copyleft ones are under a fair use policy, so it will be good for your testing

[RSATom]

@dcposch

First, the bad news: WebChimera doesn't look like it will work for now. It is flaky and just crashes the renderer process about 50% of the time I try to play a video.

What OS do you use?

[dcposch]

@RSATom Electron 0.37.5 on Ubuntu 15.10

Can't downgrade to an older version of Electron (as used, for example, by wcjs-prebuilt) because WebTorrent uses ES6 features all over the code that are only supported in the latest versions of Electron

[RSATom]

@dcposch, ah... Linux... yes, latest Electron versions has some strange compatibility issue with libvlc on Linux: https://github.com/RSATom/WebChimera.js/issues/69

I'll look what I can do with it.

[dcposch]

Thanks!

[dcposch]

424

[rom1504]

Until #424 is merged, this issue doesn't seem very resolved to me.

[feross]

I just noticed that VLC for iOS just added AC-3 and E-AC-3 support for iOS 9.3 and later. Apparently iOS, and OS X (since El Capitan) have native support now.

http://developer.dolby.com/News/Dolby_Audio_Support_on_iOS.aspx

I wonder if there's an ffmpeg build that can use CoreAudio on OS X? Would be nice to ship this for our OS X users.

I also opened an issue on Chromium about using CoreAudio when it has support.

https://bugs.chromium.org/p/chromium/issues/detail?id=611517

[feross]

In addtion to macOS, Windows is also shipping free, built-in AC3 support (as of Windows 8). For that reason, Microsoft Edge and Safari both support playing AC3 files in the <video> tag now.

So, the easiest way for WebTorrent Desktop to get support is if Chromium just added support too.

If you want to see AC3 support in WebTorrent Desktop, then star this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=611517

(Bonus: leave a comment explaining why you think support is important)

[FluorescentHallucinogen]

WebTorrent Desktop app is a native app designed for Mac, Windows, and Linux.

Why WebTorrent Desktop don't use FFMPEG (https://ffmpeg.org) for decoding media formats?

FFMPEG can be compiled for Mac, Windows, and Linux.

FFMPEG is very interesting because it is a free and open source library that supports almost any media codec and media container.

Adding native compiled binaries to the project may be not a good idea, but as I said before, WebTorrent Desktop is a not true platform independent software, because it uses Electron that supports only Mac, Windows, and Linux.

BTW, one crazy idea just came to my mind!

FFMPEG is not written in JavaScript. Luckily, there is a project called Emscripten.

Emscripten is a source-to-source compiler that runs as a back end to the LLVM compiler and produces a highly-optimizable subset of JavaScript known as asm.js that runs at near-native speed. Using Emscripten, it is possible to compile C and C++ code into JS and compile any other code that can be translated into LLVM bytecode into JS.

So it is possible to compile FFMPEG into JS.

And looks like it is possible to use FFMPEG compiled into JS as a media codec and/or media container polyfill for the HTML5 <audio>/<video> tag in a web browser to decode or realtime (on-the-fly) transcode formats that not software/hardware supported (without needing to download the entire media file before it can start playing). At least for audio only due to performance limitations.

Example 1: There is a test.htm page with <audio src="test.opus" /> tag, test.opus file encoded using Opus audio codec, but the web browser doesn't support Opus. The polyfill decodes test.opus to uncompressed test.wav (and change the tag to <audio src="test.wav" />).

Example 2: There is a test.htm page with <video src="test.mp4" /> tag, test.mp4 file encoded using H.264 video codec and AC3 audio codec, but the web browser doesn't support AC3. The polyfill decodes only AC3 audio to uncompressed audio data (PCM) (that supported by the browser) (and optionally encodes the uncompressed audio data to AAC that also supported by the browser), but not decode H.264 video (that supported by the browser natively). I.e. the polyfill produces fully working media file in the media container that supported by the browser (e.g. MP4/MKV) with video stream encoded using video codec that supported by the browser (H.264) and audio stream encoded using audio codec that supported by the browser (PCM/AAC).

See the following interesting projects that I've found on GitHub:

https://github.com/muaz-khan/ffmpeg.js https://github.com/bgrins/videoconverter.js https://github.com/brion/ogv.js https://github.com/duanyao/codecbox.js https://github.com/kagami/ffmpeg.js https://github.com/kagami/webm.js https://github.com/kazuki/video-codec.js https://github.com/kazuki/opus.js-sample https://github.com/sopel39/audioconverter.js https://github.com/kazuki/mediacodec.wasm

BTW, look at this awesome asm.js demo: http://beta.unity3d.com/jonas/DT2/ ;)

BTW, have you heard anything about WebAssembly?

WebAssembly is a new binary format for the web. It will be used for performance critical code and to compile any (LLVM-based) programming language other than JavaScript (initially mainly C/C++) directly to run on the web. It can be seen as a next step for asm.js. WebAssembly is being developed in a W3C Community Group. It's the standard of the future web!

See the following article for more details: http://2ality.com/2015/06/web-assembly.html.

So it is possible to compile FFMPEG directly into WebAssembly.

There are experimental native WebAssembly implementations for Google Chrome, Mozilla Firefox, Microsoft Edge and Apple Safari.

BTW, look at this awesome WebAssembly demo: https://webassembly.github.io/demo/ ;)

Seems it is possible to use WebAssembly in Node.js (built on V8 that supports WebAssembly) and in Electron (built on V8, Node.js and Chromium). :)

[FluorescentHallucinogen]

What you think about all that?

[feross]

Electron actually already includes ffmpeg since it's required by Chromium. The cleanest way to add support would be if Chromium added support for additional codecs to the video tag. That's why I opened this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=611517

Second best approach, IMO, is to change the ffmpeg build used by Electron to be able to play back more codecs. Or, to include something like an emscripten'ed version of ffmpeg to convert video. But things get tricky if we want to support seeking. I don't know how that works with ffmpeg. Can it convert from the middle of a video? Would we need to feed it a complete video segment -- and therefore need to understand all the different container formats, mkv, webm, mp4, etc.? Starts to look like a lot less clean of a solution...

[feross]

If you want Dolby surround sound (AC3) support in WebTorrent, please star this Chromium issue: https://bugs.chromium.org/p/chromium/issues/detail?id=611517

[mrslain]

Look at this mpv integration (PPAPI NaCl) https://github.com/Kagami/mpv.js

[FluorescentHallucinogen]

Finally, on 19.03.2017, the last of Dolby's patents over the AC-3 codec expired. You can read more about it on https://ac3freedomday.org (cached page: https://web.archive.org/web/20170401170436/https://ac3freedomday.org).

[feross]

@FluorescentHallucinogen Yes, I saw that site. Great news! Hoping that the Chrome team adds support so we can get support automatically without making any changes to Electron!

[paxter]

That won't solve the problem. The current audio is E-AC3, not AC3. These are different codecs.

[feross]

The current audio is E-AC3, not AC3

Plenty of audio is AC3, and this would indeed solve the issue for AC3 audio.

[FluorescentHallucinogen]

@paxter

The last E-AC-3 / Enhanced AC-3 / Dolby Digital Plus patent US6246345 (https://patents.google.com/patent/US6339757) only affects encoders, not decoders.

Moreover common open source implementations of the AC-3 (E-AC-3) encoder, such as ac3enc.c in ffmpeg's libavcodec, do not infringe this (incredibly vague) patent. In particular, they do not adjust bit allocation after initial initialization, and do not optimize bit allocation by psychoacoustic criteria: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/ac3enc.c#L776

[paxter]

Anything new on this? About three years and still no ac3 support. Webtorrent is still useless without that.

[mathiasvr]

The last E-AC-3 / Enhanced AC-3 / Dolby Digital Plus patent US6246345 (https://patents.google.com/patent/US6339757) only affects encoders, not decoders.

Unless I'm reading it wrong, this patent has coincidently expired today.

[ThaUnknown]

https://github.com/Muril-o/electron-chromium-codecs