No such filter: 'zscale' - Add zscale filter to binary ?

[gabrielstuff]

I'm trying to port the following ffmpeg command, which allow to map a HDR video stream to an equivalent SDR stream.

ffmpeg -i my_video.mp4 -vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -c:v libx264 -crf 17 -preset slower my_video-tonemapped.mp4

I'd like to be able to use zscale filter.

The ffmpeg command I tried to execute was :

await ffmpeg.exec([
      '-i',
      videoFilename,
      '-vf',

 'zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p',
      '-c:v',
      'libx264',
      '-crf',
      '17',
      '-preset',
      'slower',
      'my_video-tonemapped.mp4'
    ])

Which result in the following error :

[AVFilterGraph @ 0x106cdf0] No such filter: 'zscale'
App.vue:70 Error reinitializing filters!
App.vue:70 Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0
App.vue:70 [aac @ 0xe96330] Qavg: 228.297
App.vue:70 [aac @ 0xe96330] 2 frames left in the queue on closing
App.vue:70 Conversion failed!
App.vue:70 Aborted()

Thanks a lot for this amazing project.

[abernier]

Maybe^1 the ffmpeg wasm binary was not build with --enable-libzimg required for zscale filter?

If I try ffmpeg -buildconf this on @ffmpeg/ffmpeg@0.12.6 latest version, I get:

<script type="module">
import { FFmpeg } from "@ffmpeg/ffmpeg";
import { toBlobURL } from "@ffmpeg/util";

const baseURL = "/assets/core/package/dist/esm";

const ffmpeg = new FFmpeg();
await ffmpeg.load({
  coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, "text/javascript"),
  wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, "application/wasm"),
});

ffmpeg.on("log", ({ message }) => console.log(message));

await ffmpeg.exec(['-buildconf']);
</script>

new.astro:653 ffmpeg version 5.1.3 Copyright (c) 2000-2022 the FFmpeg developers
new.astro:653   built with emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.40 (5c27e79dd0a9c4e27ef2326841698cdd4f6b5784)
new.astro:653   configuration: --target-os=none --arch=x86_32 --enable-cross-compile --disable-asm --disable-stripping --disable-programs --disable-doc --disable-debug --disable-runtime-cpudetect --disable-autodetect --nm=emnm --ar=emar --ranlib=emranlib --cc=emcc --cxx=em++ --objcc=emcc --dep-cc=emcc --extra-cflags='-I/opt/include -O3 -msimd128' --extra-cxxflags='-I/opt/include -O3 -msimd128' --disable-pthreads --disable-w32threads --disable-os2threads --enable-gpl --enable-libx264 --enable-libx265 --enable-libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libopus --enable-zlib --enable-libwebp --enable-libfreetype --enable-libfribidi --enable-libass
new.astro:653   libavutil      57. 28.100 / 57. 28.100
new.astro:653   libavcodec     59. 37.100 / 59. 37.100
new.astro:653   libavformat    59. 27.100 / 59. 27.100
new.astro:653   libavdevice    59.  7.100 / 59.  7.100
new.astro:653   libavfilter     8. 44.100 /  8. 44.100
new.astro:653   libswscale      6.  7.100 /  6.  7.100
new.astro:653   libswresample   4.  7.100 /  4.  7.100
new.astro:653   libpostproc    56.  6.100 / 56.  6.100
new.astro:653 
new.astro:653   configuration:
new.astro:653     --target-os=none
new.astro:653     --arch=x86_32
new.astro:653     --enable-cross-compile
new.astro:653     --disable-asm
new.astro:653     --disable-stripping
new.astro:653     --disable-programs
new.astro:653     --disable-doc
new.astro:653     --disable-debug
new.astro:653     --disable-runtime-cpudetect
new.astro:653     --disable-autodetect
new.astro:653     --nm=emnm
new.astro:653     --ar=emar
new.astro:653     --ranlib=emranlib
new.astro:653     --cc=emcc
new.astro:653     --cxx=em++
new.astro:653     --objcc=emcc
new.astro:653     --dep-cc=emcc
new.astro:653     --extra-cflags='-I/opt/include -O3 -msimd128'
new.astro:653     --extra-cxxflags='-I/opt/include -O3 -msimd128'
new.astro:653     --disable-pthreads
new.astro:653     --disable-w32threads
new.astro:653     --disable-os2threads
new.astro:653     --enable-gpl
new.astro:653     --enable-libx264
new.astro:653     --enable-libx265
new.astro:653     --enable-libvpx
new.astro:653     --enable-libmp3lame
new.astro:653     --enable-libtheora
new.astro:653     --enable-libvorbis
new.astro:653     --enable-libopus
new.astro:653     --enable-zlib
new.astro:653     --enable-libwebp
new.astro:653     --enable-libfreetype
new.astro:653     --enable-libfribidi
new.astro:653     --enable-libass
new.astro:653 Aborted()
0

Maybe you could build your own version of ffmpeg-core.wasm?

https://github.com/ffmpegwasm/ffmpeg.wasm/blob/9110bb0a510291cff8b481a8b6f6327e98df2018/Dockerfile#L144-L157

[gabrielstuff]

Nice approach, didn't think of trying : await ffmpeg.exec(['-buildconf']);

Lets see how to build that.

[gabrielstuff]

On mac with docker v4.22.1 it failed what ever I tried. make prd-mt and make prd can't finish build and fail. Trying the make dev command.

[abernier]

Giving it a try too (on mac m1) with Docker Desktop v4.22.1 boosted:

$ docker buildx create --use
$ mkdir -p build-cache-st build-cache-mt
$ make prd EXTRA_ARGS="--cache-from=type=local,src=build-cache-st --cache-to=type=local,dest=build-cache-st,mode=max"
$ make prd-mt EXTRA_ARGS="--cache-from=type=local,src=build-cache-mt --cache-to=type=local,dest=build-cache-mt,mode=max"

but never ends... (or not patient enough)

Trying with (for --progress=plain docker logs):

$ make dev EXTRA_ARGS="--cache-from=type=local,src=build-cache-st --cache-to=type=local,dest=build-cache-st,mode=max"

it's long...

[abernier]

ok, I was constantly failing...

```sh > [ffmpeg-base 1/11] RUN embuilder build sdl2 sdl2-mt: 0.935 ports:INFO: retrieving port: sdl2 from https://github.com/libsdl-org/SDL/archive/release-2.24.2.zip 4.081 ports:INFO: unpacking port: sdl2 5.688 cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2.a" for subsequent builds) 124.1 system_libs:INFO: compiled 115 inputs 125.0 cache:INFO: - ok 125.0 embuilder:INFO: ...success. Took 02:04 mins (124.06s) 125.0 embuilder:INFO: building sdl2-mt 125.0 cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libSDL2-mt.a... (this will be cached in "/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2-mt.a" for subsequent builds) 274.2 emcc: error: '/emsdk/upstream/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_SHARED_MEMORY__=1 -Werror=implicit-function-declaration --sysroot=/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -c -g3 -Werror -O2 -I/emsdk/upstream/emscripten/cache/ports/sdl2/SDL-release-2.24.2 -pthread -I/emsdk/upstream/emscripten/cache/sysroot/include/SDL2 -matomics -mbulk-memory /emsdk/upstream/emscripten/cache/ports/sdl2/SDL-release-2.24.2/src/joystick/SDL_gamecontroller.c -o /emsdk/upstream/emscripten/cache/ports-builds/sdl2/src/joystick/SDL_gamecontroller.c.o' failed (received SIGKILL (-9)) 284.2 embuilder: error: Subprocess 33/115 failed (returned 1)! (cmdline: /emsdk/upstream/emscripten/emcc -c /emsdk/upstream/emscripten/cache/ports/sdl2/SDL-release-2.24.2/src/joystick/SDL_gamecontroller.c -o /emsdk/upstream/emscripten/cache/ports-builds/sdl2/src/joystick/SDL_gamecontroller.c.o -g -sSTRICT -Werror -O2 -I/emsdk/upstream/emscripten/cache/ports/sdl2/SDL-release-2.24.2 -sUSE_SDL=0 -pthread -I/emsdk/upstream/emscripten/cache/sysroot/include/SDL2) ------ Dockerfile:129 -------------------- 127 | # Base ffmpeg image with dependencies and source code populated. 128 | FROM emsdk-base AS ffmpeg-base 129 | >>> RUN embuilder build sdl2 sdl2-mt 130 | ADD https://github.com/FFmpeg/FFmpeg.git#$FFMPEG_VERSION /src 131 | COPY --from=x264-builder $INSTALL_DIR $INSTALL_DIR -------------------- ERROR: failed to solve: process "/bin/sh -c embuilder build sdl2 sdl2-mt" did not complete successfully: exit code: 1 make[2]: *** [build] Error 1 make[1]: *** [build-st] Error 2 make: *** [dev] Error 2 ``` So I increased Docker to the max and let it burn ~1h 🔥: but another error: ```sh ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF make[2]: *** [build] Error 1 make[1]: *** [build-st] Error 2 make: *** [dev] Error 2 ``` 😩

---

I had more chance by letting Github Actions do the build for me.

For that, I forked and created my own[^1] PR: https://github.com/abernier/ffmpeg.wasm/pull/1 and let the workflow run https://github.com/abernier/ffmpeg.wasm/actions/runs/6093877792

[^1]: from my PR's branch to my own repo's main branch, ie: abernier/ffmpeg.wasm:main <- abernier/ffmpeg.wasm:abernier-abort-signal because actions from ffmpegwasm/ffmpeg.wasm must be validated before the workflow to be run

[gabrielstuff]

Did it work ?

[abernier]

✅ yes

[abernier]

you can then even download the artifacts:

eg: ffmpeg-core

[gabrielstuff]

Ok, so my fork worked out with zimg include. After starting the command, everything looks fine but suddenly :

Aborted(OOM)

My work is here : https://github.com/gabrielstuff/ffmpeg.wasm/actions/runs/6102761096#artifacts I think it must be related to my zimg.sh script https://github.com/gabrielstuff/ffmpeg.wasm/blob/main/build/zimg.sh

Any ideas ?

[abernier]

in my last experience with ffmpegwasm, "Aborted" messages didn't mean it failed (weirdly): have you checked with readFile your file isn't there?

[gabrielstuff]

OOM means Out of Memory. Yes it fails unforunately. Tried to load the file without success. On 7 Sep 2023 at 11:31 +0200, Antoine BERNIER @.***>, wrote:

in my last experience with ffmpegwasm, "Aborted" messages didn't mean it failed (weirdly): have you checked with readFile your file isn't there? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[LostBeard]

@abernier I just started using ffmpeg.wasm this week and I am also seeing the "aborted" message even when a command completes successfully which is not what I would expect.