search

jellyfin / jellyfin-ffmpeg

FFmpeg for Jellyfin
https://jellyfin.org
Other
491 stars 129 forks source link

jellyfin-ffmpeg forces hardware-decoding when using hardware-encoding #490

Closed TheClockTwister closed 3 hours ago

TheClockTwister commented 3 hours ago

Summary

While the official ffmpeg is able to decode the source file using software end encode the output using hardware, it seems that jellyfin-ffmpeg only supports all or nothing. I was able to transcode AV1 to h64 using SW decode and HW encode using ffmpeg, but not using jellyfin-ffmpeg witht thet same arguments.

PS: If you need more info, just comment ;) Thanks!

Steps To Reproduce

Running with ffmpeg (installed via apt inside jellyfin docker container)

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i /media/Av1Test.mkv -vf 'format=nv12,hwupload' -c:v h264_vaapi -threads 0 out.mkv

Works fine and transcodes with 5x (on my machine and with HW-encoding of the h264 on Intel iGPU)

Running using jellyfin-ffmpeg

/usr/lib/jellyfin-ffmpeg/ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i /media/Av1Test.mkv -vf 'format=nv12,hwupload' -c:v h264_vaapi -threads 0 out.mkv

Gives

...
[av1 @ 0x55a1d0db3080] No support for codec av1 profile 0.
[av1 @ 0x55a1d0db3080] Failed setup for format vaapi: hwaccel initialisation returned error.
[av1 @ 0x55a1d0db3080] Your platform doesn't support hardware accelerated AV1 decoding.
[av1 @ 0x55a1d0db3080] Failed to get pixel format.
Error while decoding stream #0:0: Function not implemented
[av1 @ 0x55a1d0db3080] Missing Sequence Header.
Error while decoding stream #0:0: Invalid data found when processing input
[av1 @ 0x55a1d0db3080] Missing Sequence Header.
Error while decoding stream #0:0: Invalid data found when processing input
Error while decoding stream #0:0: Resource temporarily unavailable
    Last message repeated 800 times
[av1 @ 0x5581285fc080] Missing Sequence Header.
Error while decoding stream #0:0: Invalid data found when processing input

Expected Behavior

From the logs above, jellyfin-ffmpeg tries to use hardware to decode the AV1 source which thet GPU does not support. But the command was to only use hardware for encoding the h264 output.

I would expect it to behave the same as vanilla ffmpeg and only use it for encoding, unless specified

System

jellyfin-ffmpeg version

root@57cff1ee0071:/# /usr/lib/jellyfin-ffmpeg/ffmpeg -v
ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100

ffmpeg version

root@57cff1ee0071:/# ffmpeg --version
ffmpeg version 5.1.6-0+deb12u1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
gnattu commented 3 hours ago

ffmpeg 5.x does not have vaapi av1 at all and that is the reason why it is using software. If you use a newer version it will error out just like jellyfin-ffmpeg because most hardware acceleration api does not handle software fallbacks, and Apple VideoToolbox is the only exception. But for the AV1 case, even VideoToolbox does not expose any AV1 decoder on platforms lacking a hardware decoder, so if you run on a platform does not have hardware decoder of the codec, you don't specify -hwaccel vaapi and you just use the hardware encoder.

nyanmisaka commented 3 hours ago

In addition, there is no native AV1 decoder in FFmpeg, and the only decoder is implemented through the third-party DAV1D. And the hardware hwaccel depends on the native AV1 decoder, so it is impossible to automatically fall back from HWA to the AV1 software decoder.

TheClockTwister commented 3 hours ago

Well, 2 minute response, clear resolution... 5-star rating! 😆 Thanks for clearifying!

Indeed, this works as expected:

/usr/lib/jellyfin-ffmpeg/ffmpeg -vaapi_device /dev/dri/renderD128 -i /media/Av1Test.mkv -vf 'format=nv12,hwupload' -c:v h264_vaapi -threads 0 out.mkv