Unable to find a suitable output format for 'image-%03d.png'

[w3evolutions]

What I want

1. Send a 1s webm file to node (works)
2. Take the ArrayBuffer pass it to ffmpgegjs (works-ish, if I add -f webm to the output I do get 1 element in process.MEMFS)
3. Turn the webm into N frames (possibly 30 max) in any format: bmp, jpg, png (preferable the fastest)
4. Get an array of Uint8Array of the frames created. (out of `process object somehow?)

I'm making an assumption here, I assume that all N frames that are created should be in process.MEMFS or somewhere in the process object each as a Uint8Array ?

Nice to have

• It would be ideal to get each frame as they are generated and not have to wait for all frames to be completed.

Issue

 let process = new ffmpegjs({
    MEMFS: [{name: "test.webm", data: webmData}],
    arguments: ['-i' , 'test.webm', 'image-%03d.png']
});

Results in: Unable to find a suitable output format for 'image-%03d.png' image-%03d.png: Invalid argument

Output

ffmpeg version n4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with emcc (Emscripten gcc/clang-like replacement) 1.39.11
  configuration: --cc=emcc --ranlib=emranlib --enable-cross-compile --target-os=none --arch=x86 --disable-runtime-cpudetect --disable-asm --disable-fast-unaligned --disable-pthreads --disable-w32threads --disable-os2threads --disable-debug --disable-stripping --disable-safe-bitstream-reader --disable-all --enable-ffmpeg --enable-avcodec --enable-avformat --enable-avfilter --enable-swresample --enable-swscale --disable-network --disable-d3d11va --disable-dxva2 --disable-vaapi --disable-vdpau --enable-decoder=vp8 --enable-decoder=h264 --enable-decoder=vorbis --enable-decoder=opus --enable-decoder=mp3 --enable-decoder=aac --enable-decoder=pcm_s16le --enable-decoder=mjpeg --enable-decoder=png --enable-demuxer=matroska --enable-demuxer=ogg --enable-demuxer=mov --enable-demuxer=mp3 --enable-demuxer=wav --enable-demuxer=image2 --enable-demuxer=concat --enable-protocol=file --enable-filter=aresample --enable-filter=scale --enable-filter=crop --enable-filter=overlay --enable-filter=hstack --enable-filter=vstack --disable-bzlib --disable-iconv --disable-libxcb --disable-lzma --disable-sdl2 --disable-securetransport --disable-xlib --enable-zlib --enable-encoder=libvpx_vp8 --enable-encoder=libopus --enable-muxer=webm --enable-muxer=ogg --enable-muxer=null --enable-libopus --enable-libvpx --extra-cflags='-s USE_ZLIB=1 -I../libvpx/dist/include' --extra-ldflags=-L../libvpx/dist/lib
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
[vp8 @ 0x6aa480] Warning: not compiled with thread support, using thread emulation
[opus @ 0x6ab030] Warning: not compiled with thread support, using thread emulation
Input #0, matroska,webm, from 'test.webm':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Video: vp8, yuv420p, 640x480, SAR 1:1 DAR 4:3, 31 fps, 31 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
    Stream #0:1(eng): Audio: opus, 48000 Hz, mono, fltp (default)
[NULL @ 0x6fb6a0] Unable to find a suitable output format for 'image-%03d.png'
image-%03d.png: Invalid argument

[ecc521]

I'm looking for nearly the exact use case, and am encountering the same error. Passing -f png gets a clearer error message:

[NULL @ 0x6ab100] Requested output format 'png' is not a suitable output format

The error message acts like the PNG output format was disabled. In the compile options, I see --disable-all and --enable-decoder=png, so it's quite possible that while the PNG decoder is present, the PNG encoder is not.

[ecc521]

Can confirm. The only enabled encoders are (ffmpeg -enccoders):

V..... libvpx               libvpx VP8 (codec vp8)

A..... libopus              libopus Opus (codec opus)

Whereas my machine has a massive list, including the png encoder and decoder.

The question is, why are they disabled (file size, or compatibility), and what is the alternative. Since this tool is intended to be used in a browser, I wouldn't be surprised if PNG was omitted, assuming there is a way to stream out raw pixel data, and export it using a canvas. Only odd part is, why would the PNG decoder be included then? And why wouldn't something like BMP be supported, just as a low overhead alternative to simplify implementation?

[w3evolutions]

@ecc521 Any insight on how to enable PNG encoder?

[ecc521]

You would need to compile from source, and add the PNG flag for the encoders. Not sure if it is supported though (might not be included because it doesn't work).

I'm going to take a look at streaming the output as 24 (or 32) bit pixel data, before trying a source compile. I don't actually need png (just need image data), so it would be faster for me, and I could encode if needed.

[w3evolutions]

@ecc521 I also really only need the pixel data, I just wanted 30 (or close to) images from a 1 sec video so I can send it to tensorflow and other image tools like Jimp. I need to take N steams and merge them together into 1 image.

[ecc521]

I'm not finding a good way to dump the raw pixel data either. Checking the compile options, you'll see: --enable-demuxer=image2

However the muxer is missing. Haven't tested it yet, but it appears that a recompile will probably be needed.

Very surprised that extracting the raw frames doesn't appear to be supported by default.

[ecc521]

Ok. Source compile looks like it is going to be painful. Docker is crashing on me, and the compile might have had problems anyway (got the error in #149)

Have you got anything working?

EDIT: Compile worked this time (Docker didn't crash). I'll do some testing, and see if I can get anything working. If I can, I'll put something out.

EDIT 2: The recompiled version is failing as well - Getting the same issue with format=png, a different issue with image2, and something else not specifying, but using .png:

[AVFilterGraph @ 0x6bd310] No such filter: 'select'

Error reinitializing filters!

Failed to inject frame into filter network: Invalid argument

Error while processing the decoded data for stream #0:0

Conversion failed!

Would the error in #149 be causing this? Not an expert with make and emscripten, so I'm pretty much out of ideas at this point. Still a few more things to try.

[ecc521]

Ok. Recompile is working, although the output files are now ~30MB. Looks like something went wrong with the argument syntax between my calls in node vs shell.

If 30MB files are fine for you, just build from source, and remove the --disable-all flag from the makefile. Otherwise, you'll need to do a bit more tuning, and add the specific encoders and muxers. If you store it compressed, it's only about 5.5MB.

The performance is WAY worse than I was expecting though (would this be fixed by WebAssembly? Hardware decoding?), so native binaries might still be required for my use case. Seeing 7 fps decode on a short 1280x720 video, although this might be greatly affected by start up delays (as 1 frame was nearly 10 seconds).

[ecc521]

Here's the webm build (the main NPM script): ffmpeg-webm.js.zip

I'd suggest compiling your own, but if you need other versions, I can upload those as well.

[FrankDiao]

Here's the webm build (the main NPM script): ffmpeg-webm.js.zip

I'd suggest compiling your own, but if you need other versions, I can upload those as well.

30MB is too big for me, I want to be able to compile on demand, what options should I set? Thanks!

[ecc521]

It's been about 2 years, so I don't have any of that stuff on hand anymore.

I recommend looking at the existing build setup for this library and reading the notes I left - that may help. Otherwise, good luck.

[FrankDiao]

It's been about 2 years, so I don't have any of that stuff on hand anymore.

I recommend looking at the existing build setup for this library and reading the notes I left - that may help. Otherwise, good luck.

Thank you, I found a solution.