Proposed Encoders / Decoders Libraries

jeromewu commented 4 years ago

So far only following encoders / decoders libraries are supported:

Update at 2020/11/24

Video

[x] mp4 (x264)
[x] mp4 (x265)
[x] webm (libvpx)
[x] ogv (theora)
[x] mpeg-1 (native)
[x] mpeg-2 (native)
[x] aom (av1 / avif) (only in next version, see comments below)

Audio

[x] mp3 (lame)
[x] aac (fdk-aac)
[x] wav/wv (wavpack)
[x] ogg (vorbis)
[x] opus (opus)
[x] flac (native)

Image

[x] gif (native)
[x] webp (libwebp)

Others

[x] font files (freetype)
[x] .srt, .ass subtitle files (libass) (sample code HERE)

If you there is any library you would like to add:

If it already exists in the replies, simply give an emoji to upvote
If it doesn't exist, you can reply this issue to propose.

I will check this issue periodically to decide which library to integrate next, or identify which is NOT possible to integrate at the moment.

jeromewu commented 4 years ago

RTSP

jeromewu commented 4 years ago

x265 / h265

cyberquarks commented 4 years ago

mpeg-1

grkblood13 commented 4 years ago

mpeg-2

CrypticSignal commented 4 years ago

AAC and WAV

AAC is a common audio codec for videos, and probably the next widest supported audio codec after MP3.
WAV is a common lossless audio format.

I don't know how WASM works, but here's how you usually use encode to AAC or WAV with FFmpeg: AAC: -c:a libdfdk_aac (requires FFmpeg to be compiled with fdk-aac) or -c:a aac (the native FFmpeg AAC encoder). lidfdk_aac is preferred as it's a higher quality AAC encoder. This may be useful: https://trac.ffmpeg.org/wiki/Encode/AAC WAV: -c:a pcm_s16le, -c:a pcm_s24le or -c:a pcm_s32le, where 16, 24 and 32 denote the output audio bit depth.

polytropoi commented 4 years ago

Ogg please! It's important for game audio, because mp3 doesn't loop properly.

samwatkinson1 commented 4 years ago

AV1

Cobertos commented 4 years ago

gif

smashah commented 4 years ago

libwebp

It would be helpful if you could provide some instructions on how to implement these libs so we can make some PRs

jeromewu commented 4 years ago

Hi all, for the latest version (v0.9.3), more libraries supported has been added. Please check if it helps in your project.

@smashah It would be great if you can send PRs, for integrating new libraries, you can check this repo: https://github.com/ffmpegwasm/ffmpeg.wasm-core.

Also you can check this series of posts to learn more foundation:

mathiasrw commented 4 years ago

Thank you so much for creating this!

AVIF please! The AV1 standard is gaining support in the industry and is also used as a single frame "film" as an image format for the web: AVIF. Chrome is supporting it, cloudflare is supporting it and on https://blog.cloudflare.com/generate-avif-images-with-image-resizing/ Cloudflare gives a list of good reasons why both Intel and Apple is jumping on the AVIF (single frame AV1) wagon.

jeromewu commented 4 years ago

For AV1, in fact I have successfully integrated into ffmpeg.wasm, but there are two main issues:

It takes more than a minute to transcode an one second video
It adds 2 extra MB to ffmpeg-core.wasm

So right now I disable it as it is not usable for now. For AVIF, I will check if it is slow as well.

knaik commented 4 years ago

I would love more focus on audio side of things, I am not sure if it's covered with theora or vorbis. So Opus and Flac. I mentioned vorbis ogg because Opus can be in an ogg container.

mathiasrw commented 4 years ago

@jeromewu 60x to transcode - aw. Hard one. But epic that you have already looked into it.

Is there any chance to obtain the version with AV1 included even it not being part of the main release?

jeromewu commented 4 years ago

@mathiasrw

I have published the version with AV1 in the next / 0.9.0-alpha.2 version, you can use it with following ways:

Node

$ npm install @ffmpeg/core@next
# or
$ npm install @ffmpeg/core@0.9.0-alpha.2

Browser

const ffmpeg = createFFmpeg({
  corePath: 'https://unpkg.com/@ffmpeg/core@0.9.0-alpha.2/dist/ffmpeg-core.js',
});

jeromewu commented 4 years ago

Hey guys, new / verified libraries are updated, please check and don't forget to upgrade to latest version of ffmpeg.wasm to use them.

sejbr commented 4 years ago

Any chance for h264_nvenc and hevc_nvenc (https://developer.nvidia.com/blog/nvidia-ffmpeg-transcoding-guide/)? Both of these are great and fast combined with nvidia gpus.

jeromewu commented 4 years ago

@sejbr From what I know, it is impossible to use GPU capabilities right now, these two flags are impossible at the moment.

sejbr commented 4 years ago

You mean in web assembly? That's a shame :/.

EstebanFuentealba commented 4 years ago

Thank you for your excelent work @jeromewu 👏👏! if possible add libwebp plis 🙏

jeromewu commented 4 years ago

@smashah and @EstebanFuentealba libwebp is added in v0.9.6, please check. :smile:

pbl4845 commented 3 years ago

SRT & RIST That would be even more awesome!!

sminodonte commented 3 years ago

mjpeg

owenthereal commented 3 years ago

Great work on the library. Can we support RTMP? It would be interesting to see if it's possible to stream from the browser to YouTube or Twitch.

tudalex commented 3 years ago

Jpeg2000

videoluce commented 3 years ago

I wish that it can work with mxf codec

roger912 commented 3 years ago

flv(x265) mpeg2ts(HEVC) mpeg2ps(HEVC)

amirzenoozi commented 3 years ago

RTSP

RTSP Support is Now Available ? because you add it this issue but in other issues you said you don't support streaming yet

jeromewu commented 3 years ago

@amirzenoozi RTSP is still not supported, only the ones in the top of this issue are available. Sorry for the confusion.

maickoldxd commented 3 years ago

please add suppor for libass

Felixkruemel commented 3 years ago

@jeromewu Why is libaom-av1 in WASM implementation so slow? In normal use it's pretty okay speed wise.

Could you try making a build with libsvt-av1 or rav1e? See https://gitlab.com/AOMediaCodec/SVT-AV1 or https://github.com/xiph/rav1e Just want to see how it performs for you, maybe that's at least a way to implement AV1 support.

For faster (and better) AVIF support there's also https://github.com/kornelski/cavif-rs available based on rav1e, although I don't know if that can be easily merged into ffmpeg.

sakurayang commented 3 years ago

Is it possible to add support for Avisynth?

lovewinders commented 3 years ago

@amirzenoozi RTSP is still not supported, only the ones in the top of this issue are available. Sorry for the confusion.

Are there any plans to support RTSP streaming?

jeromewu commented 3 years ago

Libass is added in v0.9.8, feel free to check. :smile:

justintaylor-dev commented 3 years ago

Animated WebP ( libwebp_anim ) which is in the latest version of FFMPEG.

wong2 commented 3 years ago

ALAC

zhw2590582 commented 3 years ago

flv(h264+aac)

evilArsh commented 3 years ago

RTSP please

liuhetongzi commented 3 years ago

RTSP please

tolerious commented 3 years ago

RTSP please

TahaBerkay commented 3 years ago

rtsp please

Aloento commented 2 years ago

aom

Captioncoco commented 2 years ago

rtsp is not supported yet?

idranme commented 2 years ago

amv

Moebits commented 2 years ago

Can you support VP9 codec please? I was wondering why encoding some webm videos resulted in a black screen video, and now I see that the VP9 codec isn't supported here.

dimagoltsman commented 2 years ago

trying to compile with https support but getting this when running in browser:

RuntimeError: abort(Assertion failed: undefined) at Error
    at jsStackTrace (ffmpeg.js:1:48319)
    at stackTrace (ffmpeg.js:1:48487)
    at abort (ffmpeg.js:1:26617)
    at assert (ffmpeg.js:1:14046)
    at Object.createSocket (ffmpeg.js:1:116479)
    at ___sys_socketcall (ffmpeg.js:1:131668)
    at _emscripten_receive_on_main_thread_js (ffmpeg.js:1:224397)
    at ffmpeg.wasm:0xd1d2cb
    at ffmpeg.wasm:0xd1cb33
    at ffmpeg.wasm:0xd1daf6
ffmpeg.js:1 RuntimeError: abort(Assertion failed: undefined) at Error
    at jsStackTrace (ffmpeg.js:1:48319)
    at stackTrace (ffmpeg.js:1:48487)
    at abort (ffmpeg.js:1:26617)
    at assert (ffmpeg.js:1:14046)
    at Object.createSocket (ffmpeg.js:1:116479)
    at ___sys_socketcall (ffmpeg.js:1:131668)
    at _emscripten_receive_on_main_thread_js (ffmpeg.js:1:224397)
    at ffmpeg.wasm:0xd1d2cb
    at ffmpeg.wasm:0xd1cb33
    at ffmpeg.wasm:0xd1daf6
abort @ ffmpeg.js:1
___sys_socketcall @ ffmpeg.js:1
_emscripten_receive_on_main_thread_js @ ffmpeg.js:1
$func20031 @ ffmpeg.wasm:0xd1d2cb
$emscripten_current_thread_process_queued_calls @ ffmpeg.wasm:0xd1cb33
$emscripten_main_thread_process_queued_calls @ ffmpeg.wasm:0xd1daf6
(anonymous) @ ffmpeg.js:1
worker.onmessage @ ffmpeg.js:1
ffmpeg.js:1 Uncaught RuntimeError: abort(RuntimeError: abort(Assertion failed: undefined) at Error
    at jsStackTrace (ffmpeg.js:1:48319)
    at stackTrace (ffmpeg.js:1:48487)
    at abort (ffmpeg.js:1:26617)
    at assert (ffmpeg.js:1:14046)
    at Object.createSocket (ffmpeg.js:1:116479)
    at ___sys_socketcall (ffmpeg.js:1:131668)
    at _emscripten_receive_on_main_thread_js (ffmpeg.js:1:224397)
    at ffmpeg.wasm:0xd1d2cb
    at ffmpeg.wasm:0xd1cb33
    at ffmpeg.wasm:wasm-function[20048]:0xd1daf6) at Error
    at jsStackTrace (ffmpeg.js:1:48319)
    at stackTrace (ffmpeg.js:1:48487)
    at abort (ffmpeg.js:1:26617)
    at ___sys_socketcall (ffmpeg.js:1:135180)
    at _emscripten_receive_on_main_thread_js (ffmpeg.js:1:224397)
    at ffmpeg.wasm:0xd1d2cb
    at ffmpeg.wasm:0xd1cb33
    at ffmpeg.wasm:0xd1daf6
    at ffmpeg.js:1:27502
    at Worker.worker.onmessage (ffmpeg.js:1:45693)
    at abort (ffmpeg.js:1:26648)
    at ___sys_socketcall (ffmpeg.js:1:135180)
    at _emscripten_receive_on_main_thread_js (ffmpeg.js:1:224397)
    at ffmpeg.wasm:0xd1d2cb
    at ffmpeg.wasm:0xd1cb33
    at ffmpeg.wasm:0xd1daf6
    at ffmpeg.js:1:27502
    at Worker.worker.onmessage (ffmpeg.js:1:45693)
abort @ ffmpeg.js:1
___sys_socketcall @ ffmpeg.js:1
_emscripten_receive_on_main_thread_js @ ffmpeg.js:1
$func20031 @ ffmpeg.wasm:0xd1d2cb
$emscripten_current_thread_process_queued_calls @ ffmpeg.wasm:0xd1cb33
$emscripten_main_thread_process_queued_calls @ ffmpeg.wasm:0xd1daf6
(anonymous) @ ffmpeg.js:1
worker.onmessage @ ffmpeg.js:1

any idea why? ()ffmpeg -protocols does show https)