search

elFarto / nvidia-vaapi-driver

A VA-API implemention using NVIDIA's NVDEC
Other
1.15k stars 53 forks source link

libva error: /usr/lib/dri/nvidia_drv_video.so init failed #261

Closed dkon70 closed 5 months ago

dkon70 commented 7 months ago

I've done everything from the readme. But when I start Firefox, I get errors in the console: libva error: /usr/lib/dri/nvidia_drv_video.so init failed. And vaapi is not working, I get about 30% CPU usage when watching videos.

archlinux 6.6.7 kernel gtx 1050ti

$ vainfo
Trying display: wayland
Trying display: x11
libva error: vaGetDriverNames() failed with unknown libva error
vainfo: VA-API version: 1.20 (libva 2.20.1)
vainfo: Driver version: VA-API NVDEC driver [egl backend]
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
Wyacheslaw8 commented 7 months ago

Hello I have the same error, check my issue #260. May be You have to change backend to "Direct backend". In addition, my card was working well last summer with early drivers and etc(?)

elFarto commented 7 months ago

Yes, you'll need to use the direct backend. I'm going to change the default to direct as NVIDIA doesn't seem to have any desire to fix the EGLStreams bug.

Wyacheslaw8 commented 7 months ago

Yes. I am using direct backend, but it is not working for my GTX1050Ti. In another PC witn RTX3080Ti the nvidia-vaapi-driver is working fine without error. In old PC with RTX1050Ti the nvidia-vaapi-driver was working last summer 2023 (early nvidia drivers and etc). I tried to install old nvidia drivers - but it didn't work.

qumaciel commented 7 months ago

I'm having the same issue and I've managed to trace back to commit 6d3ba1de028aaa3c01966c6c28db91fafa12ecef

before commit 6d3ba1de028aaa3c01966c6c28db91fafa12ecef

NVD_LOG=1 vainfo
Trying display: wayland
Trying display: x11
libva error: vaGetDriverNames() failed with unknown libva error
       703.718491524 [9601-9601] ../src/vabackend.c:2145       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 10
       703.718495875 [9601-9601] ../src/vabackend.c:2154       __vaDriverInit_1_0 Now have 0 (0 max) instances
       703.718497109 [9601-9601] ../src/vabackend.c:2180       __vaDriverInit_1_0 Selecting Direct backend
       703.723704864 [9601-9601] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
       703.723719242 [9601-9601] ../src/direct/direct-export-buf.c:  85      direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD129
       703.723722011 [9601-9601] ../src/direct/nv-driver.c: 246            init_nvdriver Initing nvdriver...
       703.723743716 [9601-9601] ../src/direct/nv-driver.c: 264            init_nvdriver NVIDIA kernel driver version: 535.43.22, major version: 535, minor version: 43
       703.723747559 [9601-9601] ../src/direct/nv-driver.c: 271            init_nvdriver Got dev info: 100 1 2 6
vainfo: VA-API version: 1.20 (libva 2.20.1)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444_10         : VAEntrypointVLD
      VAProfileHEVCMain444_12         : VAEntrypointVLD
       703.782511734 [9601-9601] ../src/vabackend.c:2055              nvTerminate Terminating 0x5567c316e660
       703.782555337 [9601-9601] ../src/vabackend.c:2069              nvTerminate Now have 0 (0 max) instances

after commit 6d3ba1de028aaa3c01966c6c28db91fafa12ecef

NVD_LOG=1 vainfo
Trying display: wayland
Trying display: x11
libva error: vaGetDriverNames() failed with unknown libva error
       737.591774322 [11471-11471] ../src/vabackend.c:2145       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 10
       737.591778682 [11471-11471] ../src/vabackend.c:2154       __vaDriverInit_1_0 Now have 0 (0 max) instances
       737.591780049 [11471-11471] ../src/vabackend.c:2180       __vaDriverInit_1_0 Selecting Direct backend
       737.596394204 [11471-11471] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
       737.596403147 [11471-11471] ../src/direct/direct-export-buf.c:  87      direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD128
       737.596406317 [11471-11471] ../src/direct/nv-driver.c: 252            init_nvdriver Initing nvdriver...
       737.596422608 [11471-11471] ../src/direct/nv-driver.c: 270            init_nvdriver NVIDIA kernel driver version: 535.43.22, major version: 535, minor version: 43
       737.596433624 [11471-11471] ../src/direct/nv-driver.c: 220          get_device_info get_device_info failed: -1 22
       737.596439111 [11471-11471] ../src/direct/nv-driver.c: 108            nv_rm_control nv_rm_control failed: -1 0 25
       737.596442885 [11471-11471] ../src/vabackend.c:2205       __vaDriverInit_1_0 Exporter failed
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
vaInitialize failed with error code 1 (operation failed),exit
elFarto commented 7 months ago

@qumaciel I assume that's using the open source driver? I don't see a closed source package of that driver. Might take me a while to track that down as I can't run the open source driver on my old card.

qumaciel commented 7 months ago

Nope, I’m using the latest proprietary driver (vulkan beta branch) 535.43.22

It’s an Optimus laptop tough. 

Dec 20, 2023 at 15:01 by @.***:

@qumaciel https://github.com/qumaciel> I assume that's using the open source driver? I don't see a closed source package of that driver. Might take me a while to track that down as I can't run the open source driver on my old card.

— Reply to this email directly, > view it on GitHub https://github.com/elFarto/nvidia-vaapi-driver/issues/261#issuecomment-1864276364> , or > unsubscribe https://github.com/notifications/unsubscribe-auth/ACLF552SU4MCGYP6IG3HTZLYKLAPXAVCNFSM6AAAAABAX3OO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRUGI3TMMZWGQ> . You are receiving this because you were mentioned.> Message ID: > <elFarto/nvidia-vaapi-driver/issues/261/1864276364> @> github> .> com>

elFarto commented 7 months ago

Just installed that version and it works on my machine. I'm a bit surprised that commit is the one that broke it, nothing changed between them that should have broken that particular request (although a lot changed on buffer allocation, vainfo doesn't do any of that).

dkon70 commented 7 months ago

Doesn't work even with direct backend in arch. It worked just fine in Ubuntu 23.10 a month ago, so I decided to switch back to Ubuntu. And... it doesn't work anymore. With the same version of nvidia-vaapi-driver and nvidia driver. I have no idea what happened. Also I tried to downgrade to 525 nvidia driver but it had no effect.

qumaciel commented 7 months ago

Just installed that version and it works on my machine. I'm a bit surprised that commit is the one that broke it, nothing changed between them that should have broken that particular request (although a lot changed on buffer allocation, vainfo doesn't do any of that).

Somehow, the nvdGpu index mismatched my card. I've managed to get back to normal behavior using the NVD_GPU environment variable.

elFarto commented 7 months ago

Somehow, the nvdGpu index mismatched my card. I've managed to get back to normal behavior using the NVD_GPU environment variable.

I'm not entirely sure how that's possible in an Optimus laptop. Without setting any environment variables nvdGpu comes from the index of of you GPU which should be 0 as CUDA should only return 1 GPU available.

If you could add this LOG line in the direct_initExporter function that would be very helpful:

        do {
            LOG("Searching for GPU: %d %d %d", nvIdx, nvdGpu, drmIdx)
            snprintf(node, 20, "/dev/dri/renderD%d", drmIdx++);
s1nceri7y commented 6 months ago

I have the similar error.

Arch Linux 6.6.8-zen1-1-zen, rtx 3070.

I tried two packages: official libva-nvidia-driver-0.0.11-1 and aur(git) 0.0.11.r11.gf977766-1 (Sun 24 Dec 2023 03:09:46 PM CET).

Tried both backends(direct and EGL). No luck.

I also had the similar error on the same PC on manjaro.

log

```shell % LIBVA_DRIVER_NAME=nvidia NVD_LOG=1 vainfo Trying display: wayland Trying display: x11 libva error: vaGetDriverNames() failed with unknown libva error 4836.806365666 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2140 __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 10 4836.806377628 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2149 __vaDriverInit_1_0 Now have 0 (0 max) instances 4836.806381776 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2175 __vaDriverInit_1_0 Selecting Direct backend 4836.812729428 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/direct/direct-export-buf.c: 85 direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD128 4836.812738395 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 246 init_nvdriver Initing nvdriver... 4836.812762531 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 264 init_nvdriver NVIDIA kernel driver version: 545.29.06, major version: 545, minor version: 29 4836.812767340 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/direct/nv-driver.c: 271 init_nvdriver Got dev info: 900 1 2 6 vainfo: VA-API version: 1.20 (libva 2.20.1) vainfo: Driver version: VA-API NVDEC driver [direct backend] vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointVLD VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileH264Main : VAEntrypointVLD VAProfileH264High : VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileHEVCMain : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain12 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD VAProfileHEVCMain444 : VAEntrypointVLD VAProfileHEVCMain444_10 : VAEntrypointVLD VAProfileHEVCMain444_12 : VAEntrypointVLD 4836.914787574 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2050 nvTerminate Terminating 0x5625b56bd490 4836.914852618 [21321-21321] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2064 nvTerminate Now have 0 (0 max) instances ``` ``` % cat 1.132058.thd-0x0000207a [12346.589528][ctx none]==========va_TraceInitialize [12346.589533][ctx none]========== VA-API vendor string: VA-API NVDEC driver [direct backend] [12346.589534][ctx none]=========vaInitialize ret = VA_STATUS_SUCCESS, success (no error) [12346.614607][ctx none]=========vaQueryConfigProfiles ret = VA_STATUS_SUCCESS, success (no error) [12346.614613][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614615][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614617][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614619][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614622][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614624][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614627][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614629][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614635][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614637][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614639][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614643][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614646][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614648][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614650][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614652][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614656][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.614658][ctx none]=========vaQueryConfigEntrypoints ret = VA_STATUS_SUCCESS, success (no error) [12346.638789][ctx none]==========va_TraceTerminate [12346.638825][ctx none]=========vaTerminate ret = VA_STATUS_SUCCESS, success (no error) ``` ``` % LIBVA_DRIVER_NAME=nvidia VDPAU_DRIVER=nvidia NVD_BACKEND=direct ```

Firefox sees the hardware acceleration support after a while, but nevertheless uses the CPU. (~40% on 8K av1 youtube video).

screenshot

![image](https://github.com/elFarto/nvidia-vaapi-driver/assets/23638148/b2f61fb0-cdb3-400e-9053-b16d45364519)

It's also works ok on "DRM" display.

log

```shell LIBVA_DRIVER_NAME=nvidia NVD_LOG=1 vainfo --display drm Trying display: drm 5773.584293966 [25672-25672] ../src/vabackend.c:2154 __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31 5773.584302312 [25672-25672] ../src/vabackend.c:2163 __vaDriverInit_1_0 Now have 0 (0 max) instances 5773.584304637 [25672-25672] ../src/vabackend.c:2189 __vaDriverInit_1_0 Selecting Direct backend 5773.590280403 [25672-25672] ../src/direct/nv-driver.c: 254 init_nvdriver Initing nvdriver... 5773.590313316 [25672-25672] ../src/direct/nv-driver.c: 272 init_nvdriver NVIDIA kernel driver version: 545.29.06, major version: 545, minor version: 29 5773.590318375 [25672-25672] ../src/direct/nv-driver.c: 279 init_nvdriver Got dev info: 900 1 2 6 vainfo: VA-API version: 1.20 (libva 2.20.1) vainfo: Driver version: VA-API NVDEC driver [direct backend] vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointVLD VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileH264Main : VAEntrypointVLD VAProfileH264High : VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileHEVCMain : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain12 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD VAProfileHEVCMain444 : VAEntrypointVLD VAProfileHEVCMain444_10 : VAEntrypointVLD VAProfileHEVCMain444_12 : VAEntrypointVLD 5773.682803977 [25672-25672] ../src/vabackend.c:2064 nvTerminate Terminating 0x55d043bc9a30 5773.682858130 [25672-25672] ../src/vabackend.c:2078 nvTerminate Now have 0 (0 max) instances ```

UPD: export NVD_GPU="/dev/dri/renderD128" fixed for me. UPD2: it's not. the same with NVD_GPU="0". some configuration mess on my side

s1nceri7y commented 6 months ago

I spent some time learning how to debug using gdb. I'm getting the -1 here, which produces the error.

Stack-trace

![image](https://github.com/elFarto/nvidia-vaapi-driver/assets/23638148/a0ef7902-971a-45ef-9f25-ce57e668fc94)

Just in case I'll leave the strace log here.
log.txt

qumaciel commented 6 months ago

I'm not entirely sure how that's possible in an Optimus laptop. Without setting any environment variables nvdGpu comes from the index of of you GPU which should be 0 as CUDA should only return 1 GPU available.

If you could add this LOG line in the direct_initExporter function that would be very helpful:

        do {
            LOG("Searching for GPU: %d %d %d", nvIdx, nvdGpu, drmIdx)
            snprintf(node, 20, "/dev/dri/renderD%d", drmIdx++);

Sorry the delay. I think you added the logs to master, right?!

I'm back to the optimus notebook with A2000 card. I've changed the driver to regular 545.29.06 with plasma wayland session.

Here is the output without setting NVD_GPU:

❯ NVD_LOG=1 vainfo
Trying display: wayland
       136.879896494 [5068-5068] ../src/vabackend.c:2154       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 40
       136.879922385 [5068-5068] ../src/vabackend.c:2163       __vaDriverInit_1_0 Now have 0 (0 max) instances
       136.879925861 [5068-5068] ../src/vabackend.c:2189       __vaDriverInit_1_0 Selecting Direct backend
       136.888579772 [5068-5068] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
       136.888586504 [5068-5068] ../src/vabackend.c:2214       __vaDriverInit_1_0 Exporter failed
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
vaInitialize failed with error code 1 (operation failed),exit

and here is the output setting NVD_GPU=0 

❯ NVD_LOG=1 NVD_GPU=0 vainfo
Trying display: wayland
       284.635795343 [5225-5225] ../src/vabackend.c:2154       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 40
       284.635802803 [5225-5225] ../src/vabackend.c:2163       __vaDriverInit_1_0 Now have 0 (0 max) instances
       284.635803968 [5225-5225] ../src/vabackend.c:2189       __vaDriverInit_1_0 Selecting Direct backend
       284.640896437 [5225-5225] ../src/direct/direct-export-buf.c:  66      direct_initExporter Searching for GPU: 0 0 128
       284.640910869 [5225-5225] ../src/direct/direct-export-buf.c:  88      direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD128
       284.640912847 [5225-5225] ../src/direct/nv-driver.c: 254            init_nvdriver Initing nvdriver...
       284.640927246 [5225-5225] ../src/direct/nv-driver.c: 272            init_nvdriver NVIDIA kernel driver version: 545.29.06, major version: 545, minor version: 29
       284.640929423 [5225-5225] ../src/direct/nv-driver.c: 279            init_nvdriver Got dev info: 100 1 2 6
vainfo: VA-API version: 1.20 (libva 2.20.1)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444_10         : VAEntrypointVLD
      VAProfileHEVCMain444_12         : VAEntrypointVLD
       284.707877667 [5225-5225] ../src/vabackend.c:2064              nvTerminate Terminating 0x56481cb5ecb0
       284.707934876 [5225-5225] ../src/vabackend.c:2078              nvTerminate Now have 0 (0 max) instances
elFarto commented 6 months ago

I spent some time learning how to debug using gdb. I'm getting the -1 here, which produces the error.

DRI3 doesn't work on the NVIDIA (at least not the Open function). It should fall back to DRI2, however the libva team have made changes to how their X11 backend selects a driver which has broken the auto driver detection. Setting LIBVA_DRIVER_NAME will force it to use the correct driver.

As for why Firefox doesn't use it, that might be the ffvpx issue. Check if media.ffvpx.enabled is set to true in about:config.

elFarto commented 6 months ago 
❯ NVD_LOG=1 vainfo
Trying display: wayland
       136.879896494 [5068-5068] ../src/vabackend.c:2154       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 40
       136.879922385 [5068-5068] ../src/vabackend.c:2163       __vaDriverInit_1_0 Now have 0 (0 max) instances
       136.879925861 [5068-5068] ../src/vabackend.c:2189       __vaDriverInit_1_0 Selecting Direct backend
       136.888579772 [5068-5068] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
       136.888586504 [5068-5068] ../src/vabackend.c:2214       __vaDriverInit_1_0 Exporter failed
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
vaInitialize failed with error code 1 (operation failed),exit

Ok, so this is an issue with the driver running under Wayland on an Optimus system. The issue here is that we're forcing libva to load the NVIDIA VA-API driver, but passing it the DRM fd that the Wayland compositor is using, which in this case is the Intel one.

Because we're passed in an existing DRM fd, we can't really go searching for a different card like we do under X11 as the calling application (Firefox) is expecting to be able to import the exported frames into that fd. We can't currently export from NVIDIA and import into Intel. In theory this is possible, but I'm not sure I'll be able to figure out how to get the NVIDIA kernel driver to create memory that can be read by the Intel card.

In theory the correct way around this (apart from making this driver work with Optimus) is to run Firefox on the NVIDIA card. This should cause Firefox the initialise libva with the correct DRM fd, which would cause libva to pick the right driver, and everything should work. However Firefox really doesn't like this setup, as there are various places in the code that use the 'default' device, which would be the one the compositor is running on. This is really a Firefox issue as there's nothing I can do with it.

mirh commented 5 months ago

It should fall back to DRI2, however the libva team have made changes to how their X11 backend selects a driver which has broken the auto driver detection.

I don't know how much you could consider the hardcoded list of names that they always had as "autodetection", anyhow even if you specify LIBVA_DRI3_DISABLE=1 (meaning that nothing but DRI2 ever runs) it doesn't work. The first obvious reason is that va_DRI2_GetDriverNames doesn't use VA_DRM_GetDriverNames (so nothing else look for the nvidia string) but even after jerry-rigging all the names that I could come up with it still continued to fail.