nvCreateContext CUDA ERROR 'invalid argument' when trying decode using ffmpeg

jackdaw213 commented 9 months ago

I'm using the driver for Firefox hardware acceleration but it's not working so I tried it with ffmpeg first but it does not work either

Operating System: Arch Linux
KDE Plasma Version: 5.27.9
KDE Frameworks Version: 5.112.0
Qt Version: 5.15.11
Kernel Version: 6.6.2-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 12 × Intel® Core™ i5-10400 CPU @ 2.90GHz
Memory: 23.3 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 3080/PCIe/SSE2
The driver was installed via AUR Firefox infos:

NVD_LOG=1 vainfo:

Trying display: wayland
Trying display: x11
      3049.509766559 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2140       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 10
      3049.509774859 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2149       __vaDriverInit_1_0 Now have 0 (0 max) instances
      3049.509779000 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2175       __vaDriverInit_1_0 Selecting Direct backend
      3049.516303307 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
      3049.516322142 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/direct/direct-export-buf.c:  85      direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD129
      3049.516324674 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 246            init_nvdriver Initing nvdriver...
      3049.516344727 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 264            init_nvdriver NVIDIA kernel driver version: 545.29.06, major version: 545, minor version: 29
      3049.516347852 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 271            init_nvdriver Got dev info: 100 1 2 6
vainfo: VA-API version: 1.20 (libva 2.20.0)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444_10         : VAEntrypointVLD
      VAProfileHEVCMain444_12         : VAEntrypointVLD
      3049.597582041 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2050              nvTerminate Terminating 0x5646cd8f1980
      3049.597655751 [22880-22880] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2064              nvTerminate Now have 0 (0 max) instances

ffmpeg -hwaccels:

ffmpeg version n6.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (GCC) 20230801
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Hardware acceleration methods:
vdpau
cuda
vaapi
qsv
drm
opencl
vulkan

NVD_LOG=1 ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -i input.mp4 -c:v libx264 -crf 20 output.mp4:

ffmpeg version n6.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (GCC) 20230801
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:05.78, start: 0.000000, bitrate: 376 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3440x1440 [SAR 1:1 DAR 43:18], 359 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 libx264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 2 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      2978.160644983 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2140       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
      2978.160654785 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2149       __vaDriverInit_1_0 Now have 0 (0 max) instances
      2978.160657770 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2175       __vaDriverInit_1_0 Selecting Direct backend
      2978.167301873 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 246            init_nvdriver Initing nvdriver...
      2978.167327645 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 264            init_nvdriver NVIDIA kernel driver version: 545.29.06, major version: 545, minor version: 29
      2978.167331819 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 271            init_nvdriver Got dev info: 100 1 2 6
      2978.224421296 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1398      nvQueryImageFormats In nvQueryImageFormats
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
      2978.255020786 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 637           nvCreateConfig got profile: 7 with 0 attributes
      2978.255034121 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) (nil) 0
      2978.255036600 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) 0x7fb5742130c0 8
      2978.255964347 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1821 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
      2978.255975757 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) (nil) 32693
      2978.255978245 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) 0x7fb5742130c0 8
      2978.257400882 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1821 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
      2978.257418602 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420fa60)
      2978.257422617 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1492            nvDeriveImage In nvDeriveImage
      2978.257426445 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb574213a20)
      2978.257429343 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420ac20)
      2978.257432085 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420b020)
      2978.257434605 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420b420)
      2978.257437164 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420b820)
      2978.257439732 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420bc20)
      2978.257442264 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420c020)
      2978.257444912 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420c420)
      2978.257447414 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420c820)
      2978.257449961 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420cc20)
      2978.257452695 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420d020)
      2978.257455210 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420d420)
      2978.257457720 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420d820)
      2978.257460376 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420dc20)
      2978.257462924 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420e020)
      2978.257465667 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420e420)
      2978.257468707 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57420e6e0)
      2978.257471751 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb5740581e0)
      2978.257474338 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb5740585e0)
      2978.257476963 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb5740589e0)
      2978.257479529 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb574058de0)
      2978.257482086 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb5740591e0)
      2978.257484679 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb5740595e0)
      2978.257487308 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb5740599e0)
      2978.257489837 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb574059f60)
      2978.257492540 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405a360)
      2978.257495029 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405a760)
      2978.257497542 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405ab60)
      2978.257500139 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405af60)
      2978.257502656 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405b360)
      2978.257505137 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405b760)
      2978.257507722 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 3440x1440, format 1 (0x7fb57405bb60)
      2978.280906191 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 637           nvCreateConfig got profile: 7 with 0 attributes
      2978.280926133 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) (nil) 0
      2978.280932208 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) 0x7fb57420e4c0 8
      2978.282309619 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1821 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
      2978.282326781 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 984          nvCreateContext with 33 render targets, 33 surfaces, at 3440x1440
      2978.282405479 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1037          nvCreateContext CUDA ERROR 'invalid argument' (1)

[h264 @ 0x55c80ce39bc0] Failed to create decode context: 2 (resource allocation failed).
[h264 @ 0x55c80ce39bc0] Failed setup for format vaapi: hwaccel initialisation returned error.
      2978.282432105 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405bb60)
      2978.282436121 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405b760)
      2978.282439348 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405b360)
      2978.282441996 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405af60)
      2978.282444640 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405ab60)
      2978.282447286 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405a760)
      2978.282449642 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57405a360)
      2978.282452190 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb574059f60)
      2978.282454743 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb5740599e0)
      2978.282457433 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb5740595e0)
      2978.282460443 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb5740591e0)
      2978.282463972 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb574058de0)
      2978.282466915 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb5740589e0)
      2978.282470024 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb5740585e0)
      2978.282472900 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb5740581e0)
      2978.282474983 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420e6e0)
      2978.282477572 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420e420)
      2978.282479826 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420e020)
      2978.282481721 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420dc20)
      2978.282483761 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420d820)
      2978.282485685 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420d420)
      2978.282488168 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420d020)
      2978.282490156 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420cc20)
      2978.282492284 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420c820)
      2978.282495134 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420c420)
      2978.282497317 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420c020)
      2978.282499431 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420bc20)
      2978.282501565 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420b820)
      2978.282504004 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420b420)
      2978.282505935 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420b020)
      2978.282508380 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420ac20)
      2978.282510961 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb574213a20)
      2978.282512862 [22615-22620] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7fb57420fa60)
[libx264 @ 0x55c80bdb7e00] using SAR=1/1
[libx264 @ 0x55c80bdb7e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x55c80bdb7e00] profile High, level 5.2, 4:2:0, 8-bit
[libx264 @ 0x55c80bdb7e00] 264 - core 164 r3108 31e19f9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3440x1440 [SAR 1:1 DAR 43:18], q=2-31, 60 fps, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 aac
[out#0/mp4 @ 0x55c80bd74240] video:248kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.175495%
frame=  347 fps= 71 q=-1.0 Lsize=     260kB time=00:00:05.73 bitrate= 371.8kbits/s speed=1.17x    
[libx264 @ 0x55c80bdb7e00] frame I:2     Avg QP:16.06  size: 74614
[libx264 @ 0x55c80bdb7e00] frame P:87    Avg QP:14.25  size:   541
[libx264 @ 0x55c80bdb7e00] frame B:258   Avg QP:12.67  size:   222
[libx264 @ 0x55c80bdb7e00] consecutive B-frames:  0.9%  0.0%  0.0% 99.1%
[libx264 @ 0x55c80bdb7e00] mb I  I16..4: 27.3% 65.6%  7.1%
[libx264 @ 0x55c80bdb7e00] mb P  I16..4:  0.0%  0.1%  0.0%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:99.7%
[libx264 @ 0x55c80bdb7e00] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.7%  0.0%  0.0%  direct: 0.0%  skip:99.3%  L0:68.6% L1:31.4% BI: 0.0%
[libx264 @ 0x55c80bdb7e00] 8x8 transform intra:64.6% inter:55.7%
[libx264 @ 0x55c80bdb7e00] coded y,uvDC,uvAC intra: 8.4% 5.4% 5.0% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x55c80bdb7e00] i16 v,h,dc,p: 71% 25%  4%  0%
[libx264 @ 0x55c80bdb7e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 71%  4% 22%  0%  0%  1%  0%  1%  1%
[libx264 @ 0x55c80bdb7e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 46% 22%  8%  2%  5%  5%  4%  4%  4%
[libx264 @ 0x55c80bdb7e00] i8c dc,h,v,p: 92%  4%  4%  0%
[libx264 @ 0x55c80bdb7e00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x55c80bdb7e00] ref P L0: 76.4%  4.9%  9.0%  9.8%
[libx264 @ 0x55c80bdb7e00] ref B L0: 50.9% 49.0%  0.2%
[libx264 @ 0x55c80bdb7e00] ref B L1: 99.8%  0.2%
[libx264 @ 0x55c80bdb7e00] kb/s:350.60
[aac @ 0x55c80d8e8540] Qavg: 65536.000
      2983.122702151 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2050              nvTerminate Terminating 0x55c80be6b0e0
      2983.122908390 [22615-22615] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2064              nvTerminate Now have 0 (0 max) instances

rkoot commented 9 months ago

This driver provides only hw accel in firefox. So it won't work in chome, ffmpeg etc etc etc.

elFarto commented 9 months ago

It should work in ffmpeg, as that's how Firefox access VA-API.

The error you're getting is odd, a looks like a fundamental issue and I'm not sure what's causing it. Could you try playing back a smaller video (resolution wise)?

jackdaw213 commented 9 months ago

So I downscaled the original video to 1280x720, and the same issue still happens

ffmpeg version n6.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (GCC) 20230801
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:05.78, start: 0.000000, bitrate: 140 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 43:32 DAR 43:18], 123 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 libx264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 2 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
       530.997050546 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2140       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
       530.997059710 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2149       __vaDriverInit_1_0 Now have 0 (0 max) instances
       530.997061568 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2175       __vaDriverInit_1_0 Selecting Direct backend
       531.004613680 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 246            init_nvdriver Initing nvdriver...
       531.004658300 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 264            init_nvdriver NVIDIA kernel driver version: 545.29.06, major version: 545, minor version: 29
       531.004662255 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 271            init_nvdriver Got dev info: 100 1 2 6
       531.063696566 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1398      nvQueryImageFormats In nvQueryImageFormats
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
       531.095578704 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 637           nvCreateConfig got profile: 7 with 0 attributes
       531.095591056 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) (nil) 0
       531.095593487 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) 0x7f1b501ffbc0 8
       531.096575316 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1821 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
       531.096583321 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) (nil) 32539
       531.096585873 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) 0x7f1b501ffbc0 8
       531.097760195 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1821 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
       531.097780430 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502066d0)
       531.097784501 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1492            nvDeriveImage In nvDeriveImage
       531.097788556 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502002a0)
       531.097791552 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502009b0)
       531.097794638 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b50200de0)
       531.097797839 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502011e0)
       531.097800518 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502015e0)
       531.097803146 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502019e0)
       531.097806234 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b50201de0)
       531.097810187 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502021e0)
       531.097814054 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502025e0)
       531.097817021 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502029e0)
       531.097819977 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b50202de0)
       531.097823962 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502031e0)
       531.097826557 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502035e0)
       531.097829279 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b502039e0)
       531.097832395 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b50203ba0)
       531.097835888 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004d660)
       531.097838736 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004db60)
       531.097841480 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004df60)
       531.097844077 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004e360)
       531.097846746 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004e760)
       531.097849406 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004eb60)
       531.097851967 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004ef60)
       531.097854618 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004f360)
       531.097857391 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004f760)
       531.097860020 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b5004fce0)
       531.097862691 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b500500e0)
       531.097865307 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b500504e0)
       531.097867838 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b500508e0)
       531.097870505 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b50050ce0)
       531.097873110 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b500510e0)
       531.097875682 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b500514e0)
       531.097878295 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 920        nvCreateSurfaces2 Creating surface 1280x720, format 1 (0x7f1b500518e0)
       531.122445693 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 637           nvCreateConfig got profile: 7 with 0 attributes
       531.122457595 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) (nil) 0
       531.122460398 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1754 nvQuerySurfaceAttributes with 4 (8) 0x7f1b50200680 8
       531.123364327 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1821 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
       531.123372511 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 984          nvCreateContext with 33 render targets, 33 surfaces, at 1280x720
       531.123405872 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1037          nvCreateContext CUDA ERROR 'invalid argument' (1)

[h264 @ 0x55f12e09c3c0] Failed to create decode context: 2 (resource allocation failed).
[h264 @ 0x55f12e09c3c0] Failed setup for format vaapi: hwaccel initialisation returned error.
       531.123433452 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b500518e0)
       531.123436821 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b500514e0)
       531.123439459 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b500510e0)
       531.123442599 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b50050ce0)
       531.123445887 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b500508e0)
       531.123448058 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b500504e0)
       531.123450310 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b500500e0)
       531.123452373 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004fce0)
       531.123454295 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004f760)
       531.123456448 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004f360)
       531.123458439 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004ef60)
       531.123460418 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004eb60)
       531.123462502 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004e760)
       531.123465827 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004e360)
       531.123470559 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004df60)
       531.123478301 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004db60)
       531.123482153 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b5004d660)
       531.123484588 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b50203ba0)
       531.123486662 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502039e0)
       531.123488640 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502035e0)
       531.123491397 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502031e0)
       531.123493467 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b50202de0)
       531.123495128 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502029e0)
       531.123497219 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502025e0)
       531.123499180 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502021e0)
       531.123501080 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b50201de0)
       531.123503048 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502019e0)
       531.123504956 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502015e0)
       531.123507032 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502011e0)
       531.123509749 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b50200de0)
       531.123511845 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502009b0)
       531.123513830 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502002a0)
       531.123515926 [6621-6628] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 954        nvDestroySurfaces Destroying surface -1 (0x7f1b502066d0)
[libx264 @ 0x55f12d3bdcc0] using SAR=43/32
[libx264 @ 0x55f12d3bdcc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x55f12d3bdcc0] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 0x55f12d3bdcc0] 264 - core 164 r3108 31e19f9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 43:32 DAR 43:18], q=2-31, 60 fps, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 aac
[out#0/mp4 @ 0x55f12d384240] video:95kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 10.788947%
frame=  347 fps=319 q=-1.0 Lsize=     107kB time=00:00:05.73 bitrate= 153.1kbits/s speed=5.28x    
[libx264 @ 0x55f12d3bdcc0] frame I:2     Avg QP:17.89  size: 30800
[libx264 @ 0x55f12d3bdcc0] frame P:88    Avg QP:14.57  size:   203
[libx264 @ 0x55f12d3bdcc0] frame B:257   Avg QP:12.67  size:    67
[libx264 @ 0x55f12d3bdcc0] consecutive B-frames:  0.9%  1.2%  0.0% 98.0%
[libx264 @ 0x55f12d3bdcc0] mb I  I16..4: 18.4% 63.0% 18.6%
[libx264 @ 0x55f12d3bdcc0] mb P  I16..4:  0.0%  0.1%  0.0%  P16..4:  0.3%  0.1%  0.0%  0.0%  0.0%    skip:99.5%
[libx264 @ 0x55f12d3bdcc0] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  1.1%  0.0%  0.0%  direct: 0.0%  skip:98.9%  L0:62.0% L1:38.0% BI: 0.0%
[libx264 @ 0x55f12d3bdcc0] 8x8 transform intra:61.7% inter:47.5%
[libx264 @ 0x55f12d3bdcc0] coded y,uvDC,uvAC intra: 13.3% 10.5% 9.2% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x55f12d3bdcc0] i16 v,h,dc,p: 47% 42% 11%  0%
[libx264 @ 0x55f12d3bdcc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 82%  6% 11%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x55f12d3bdcc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 52% 22%  8%  1%  3%  4%  2%  4%  3%
[libx264 @ 0x55f12d3bdcc0] i8c dc,h,v,p: 87%  7%  6%  0%
[libx264 @ 0x55f12d3bdcc0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x55f12d3bdcc0] ref P L0: 89.5%  3.9%  5.9%  0.7%
[libx264 @ 0x55f12d3bdcc0] ref B L0: 33.6% 66.4%  0.0%
[libx264 @ 0x55f12d3bdcc0] ref B L1: 99.8%  0.2%
[libx264 @ 0x55f12d3bdcc0] kb/s:133.73
[aac @ 0x55f12eb4ab00] Qavg: 65536.000
       532.161915473 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2050              nvTerminate Terminating 0x55f12d3be6b0
       532.162055940 [6621-6621] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2064              nvTerminate Now have 0 (0 max) instances

elFarto commented 8 months ago

It looks like 33 surfaces is exactly 1 too many for the driver to handle. I've added some code to master to limit the surface count to 32 which should let it run, although it may not run correctly if ffmpeg actually tries to use all those surfaces (there's no way it actually needs that many to decode the video).

jackdaw213 commented 2 months ago

@elFarto FFmpeg is working fine now, thank you for your work. However, Firefox (Nightly 129) somehow needs the MOZ_DISABLE_RDD_SANDBOX flag set to either 0 or 1 for HW decoding to work. This is odd since RDD should not be the problem for VAAPI anymore and RDD is on by default so MOZ_DISABLE_RDD_SANDBOX=0 should not change anything right ? But without it, no HW decoding

NVD_LOG=1 firefox-nightly

4851.971784688 [62440-62453] ../src/vabackend.c: 141                     init ERROR: Potential Firefox sandbox detected, failing to init!
4851.971797487 [62440-62453] ../src/vabackend.c: 142                     init If running in Firefox, set env var MOZ_DISABLE_RDD_SANDBOX=1 to disable sandbox.
4851.971822937 [62440-62453] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4851.971825982 [62440-62453] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 0 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4869.402930588 [62440-62742] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4869.402939467 [62440-62742] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 1 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4869.576256505 [62440-62748] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4869.576264999 [62440-62748] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 2 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4869.752926316 [62440-62756] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4869.752934401 [62440-62756] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 3 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4869.796281288 [62440-62760] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4869.796291997 [62440-62760] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 4 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4869.852938573 [62440-62764] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4869.852947055 [62440-62764] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 5 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4869.952920904 [62440-62768] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4869.952929050 [62440-62768] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 6 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
4875.159599303 [62440-62842] ../src/vabackend.c:2187       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
4875.159608136 [62440-62842] ../src/vabackend.c:2196       __vaDriverInit_1_0 Now have 7 (0 max) instances
libva error: /usr/lib/dri/nvidia_drv_video.so init failed

MOZ_DISABLE_RDD_SANDBOX=0 NVD_LOG=1 firefox-nightly

This will work, with MOZ_DISABLE_RDD_SANDBOX be 0 or 1

jackdaw213 commented 2 months ago

I forgot to mention but if I switch the display output to my Intel iGPU, Firefox HW decoding on the iGPU works just fine without RDD flag

elFarto / nvidia-vaapi-driver

nvCreateContext CUDA ERROR 'invalid argument' when trying decode using ffmpeg #259