search

bilibili / flv.js

HTML5 FLV Player
Apache License 2.0
22.89k stars 3.39k forks source link

WebM Remuxer #141

Closed htfy96 closed 7 years ago

htfy96 commented 7 years ago

Currently I'm working on a CEF(Chromium Embedded Framework)-based framework which is a popular choice of hundreds of applications including QQ, Unreal Engine, Minecraft, etc..

However, these applications are in a dilemma on online video: CEF does not support proprietary formats including mp3/mp4 due to licensing issues. This is also the case of Chromium. Current workarounds include:

Our application relies heavily on Flash Player to play videos, and the number of Flash-related bugs tends to grow rapidly in recent months. If WebM encoder could be implemented it would be a great benefit to all CEF users.

Here is a list of CEF's supported media formats from html5test:

There already exists a WebM encoder demo in Javascript based on ffmpeg.js compiled by Emscripten. So I was wondering whether WebM remuxer could be implemented in this project. I'm willing to contribute to this project at part-time/full-time if it is feasible to do this under current architecture.

xqq commented 7 years ago

I think there may be existing WebM / Matroska muxer in JavaScript, but not sure for now. If not, It's worth considering to use emscripten to compile some standalone C/C++ muxer into asm.js. It will be a heavy work (as well as mp4 format) to implement by ourselves.

It seems that you still want to use H264 as video codec, right?

Strictly, WebM container only accepts VPx as video codec, although it's based on Matroska.

Chrome accepts mkv file with H264 codec, but It has no guarantee for WebM segments under Media Source Extensions.

As far as I remember, when chrome plays single file directly through video tag, The video is demuxed by ffmpeg, so mkv file with H264 codec is supported;

When playback through MSE as src, The container format processor is entirely & manually implemented by the chromium authors. In the WebM demuxer, only VPx codec is supported.

BTW, I think CEF (i.e. Chromium) completely disabled H264 codec. Further checking is needed.

So if you don't recompile CEF/electron with H264/MP4 support, only WebM with VP8/VP9 can work.

htfy96 commented 7 years ago
  1. asm.js is the right orientation, I agree. ffmpeg.js actually uses such technique.
  2. We want to convert H.264 to VP8 on the fly. Unfortunately according to experiments conducted last night, this could be extremely slow(700Kbps per worker on a high-end laptop), so this proposal might be infeasible at this moment.
  3. IIRC, I've seen some projects with WebM as MSE source(?)

Maybe we have to hack into JS and perform the conversion on C++ side 😞

xqq commented 7 years ago

MSE is unnecessary because single webm file can satisfy.

At present it is a bit hard to do VPx encoding into high quality. Youtube is the best VP9 cloud transcoder 233333