search

elFarto / nvidia-vaapi-driver

A VA-API implemention using NVIDIA's NVDEC
Other
1.15k stars 53 forks source link

An issue may solve the situation to use this library on Chrome/Chromium. #274

Open LeviMarvin opened 5 months ago

LeviMarvin commented 5 months ago

Environment: CPU: Intel Core i7 13700H GPU (In about system): Mesa Intel(R) Graphics (On-Demand) GPU1: Intel Alderlake_p (Gen12) (RPL-P) [Intel Iris Xe Graphics] GPU2: NVIDIA GeForce RTX 4060 Laptop 8GB

Desktop: Debian Testing + KDE + X11

Run Chrome with the command:

NVD_LOG=1 VK_DRIVER_FILES=/usr/share/vulkan/icd.d/nvidia_icd.json LIBVA_DRIVER_NAME=nvidia google-chrome --use-angle=gl --use-gl=angle --ignore-gpu-blocklist --enable-features=VaapiVideoDecodeLinuxGL,VaapiVideoEncoder,VaapiOnNvidiaGPUs  --disable-features=UseChromeOSDirectVideoDecoder --disable-gpu-driver-bug-workarounds --enable-gpu-rasterization --enable-zero-copy

And I modified the function so that the library could run on multi-GPU mode:

static bool direct_initExporter(NVDriver *drv) {
    //this is only needed to see errors in firefox
    static const EGLAttrib debugAttribs[] = {EGL_DEBUG_MSG_WARN_KHR, EGL_TRUE, EGL_DEBUG_MSG_INFO_KHR, EGL_TRUE, EGL_NONE};
    const PFNEGLDEBUGMESSAGECONTROLKHRPROC eglDebugMessageControlKHR = (PFNEGLDEBUGMESSAGECONTROLKHRPROC) eglGetProcAddress("eglDebugMessageControlKHR");
    eglDebugMessageControlKHR(debug, debugAttribs);

    //make sure we have a drm fd
    if (drv->drmFd == -1) {
        LOG("We have NOT a drm fd.");
test:
        int nvdGpu = drv->cudaGpuId;
        if (nvdGpu == -1) {
            // The default GPU is the first one we find.
            nvdGpu = 0;
        }

        int fd;
        int nvIdx = 0;
        uint8_t drmIdx = 128;
        char node[20] = {0, };
        do {
            LOG("Searching for GPU: %d %d %d", nvIdx, nvdGpu, drmIdx)
            snprintf(node, 20, "/dev/dri/renderD%d", drmIdx++);
            fd = open(node, O_RDWR|O_CLOEXEC);
            if (fd == -1) {
                LOG("Unable to find NVIDIA GPU %d", nvdGpu);
                return false;
            }

            if (!isNvidiaDrmFd(fd, true) || !checkModesetParameterFromFd(fd)) {
                close(fd);
                continue;
            }

            if (nvIdx != nvdGpu) {
                close(fd);
                nvIdx++;
                continue;
            }
            break;
        } while (drmIdx < 128 + 16);

        drv->drmFd = fd;
        LOG("Found NVIDIA GPU %d at %s", nvdGpu, node);
    } else {
        LOG("We have a drm fd.");
        bool isNvDrmFd = isNvidiaDrmFd(drv->drmFd, true);
        bool resultCheckModesetParameterFromFd = checkModesetParameterFromFd(drv->drmFd);
        LOG("drv->drmFd: %d, isNvDrmFd: %d", drv->drmFd, isNvDrmFd);
        LOG("resultCheckModesetParameterFromFd: %d", resultCheckModesetParameterFromFd);

        if (!isNvDrmFd) {
            goto test;
        }

        if (!isNvDrmFd || !resultCheckModesetParameterFromFd) {
            return false;
        }

        //dup it so we can close it later and not effect firefox
        drv->drmFd = dup(drv->drmFd);
    }

    const bool ret = init_nvdriver(&drv->driverContext, drv->drmFd);

    //TODO this isn't really correct as we don't know if the driver version actually supports importing them
    //but we don't have an easy way to find out.
    drv->supports16BitSurface = true;
    drv->supports444Surface = true;
    findGPUIndexFromFd(drv);

    return ret;
}

And there is an error from Chrome:

[2006932:2007376:0210/185127.505732:ERROR:vaapi_video_decoder.cc(1249)] failed Initialize()ing the frame pool

Console log output:

    168257.172564446 [2006932-2006932] ../src/vabackend.c:2154       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
    168257.172568862 [2006932-2006932] ../src/vabackend.c:2163       __vaDriverInit_1_0 Now have 0 (0 max) instances
    168257.172569851 [2006932-2006932] ../src/vabackend.c:2189       __vaDriverInit_1_0 Selecting Direct backend
    168257.176688328 [2006932-2006932] ../src/direct/direct-export-buf.c:  92      direct_initExporter We have a drm fd.
    168257.176693553 [2006932-2006932] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
    168257.176695022 [2006932-2006932] ../src/direct/direct-export-buf.c:  95      direct_initExporter drv->drmFd: 26, isNvDrmFd: 0
    168257.176695690 [2006932-2006932] ../src/direct/direct-export-buf.c:  96      direct_initExporter resultCheckModesetParameterFromFd: 1
    168257.176696294 [2006932-2006932] ../src/direct/direct-export-buf.c:  68      direct_initExporter Searching for GPU: 0 0 128
    168257.176724320 [2006932-2006932] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
    168257.176728480 [2006932-2006932] ../src/direct/direct-export-buf.c:  68      direct_initExporter Searching for GPU: 0 0 129
    168257.176732355 [2006932-2006932] ../src/direct/direct-export-buf.c:  90      direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD129
    168257.176733104 [2006932-2006932] ../src/direct/nv-driver.c: 259            init_nvdriver Initing nvdriver...
    168257.176743302 [2006932-2006932] ../src/direct/nv-driver.c: 277            init_nvdriver NVIDIA kernel driver version: 545.23.08, major version: 545, minor version: 23
    168257.176744777 [2006932-2006932] ../src/direct/nv-driver.c: 284            init_nvdriver Got dev info: 100 1 2 6
    168257.257181623 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 6 == 4
    168257.257189695 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 6 with 1 attributes
    168257.257191842 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.257193612 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 4 (8) (nil) -1431655766
    168257.257194653 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 4 (8) 0xa2c013578e0 8
    168257.258123929 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
    168257.258133339 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 6 with 0 attributes
    168257.258144455 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 7 == 4
    168257.258145334 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 7 with 1 attributes
    168257.258146528 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.258147412 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 4 (8) (nil) -1431655766
    168257.258148309 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 4 (8) 0xa2c013578e0 8
    168257.259631941 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
    168257.259636860 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 7 with 0 attributes
    168257.259642289 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 13 == 4
    168257.259643082 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 13 with 1 attributes
    168257.259643831 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.259644757 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 4 (8) (nil) -1431655766
    168257.259645621 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 4 (8) 0xa2c013578e0 8
    168257.260581398 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
    168257.260585973 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 13 with 0 attributes
    168257.260591474 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 17 == 8
    168257.260592516 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 17 with 1 attributes
    168257.260593233 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.260594268 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 8 (8) (nil) -1431655766
    168257.260595048 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 8 (8) 0xa2c013578e0 8
    168257.261673455 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 144 - 8192, height: 144 - 8192
    168257.261678580 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 17 with 0 attributes
    168257.261686539 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 14 == 9
    168257.261688198 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 14 with 1 attributes
    168257.261689398 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.261690571 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 9 (8) (nil) -1431655766
    168257.261691387 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 9 (8) 0xa2c013578e0 8
    168257.262704952 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 48 - 4096, height: 16 - 4096
    168257.262709719 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 14 with 0 attributes
    168257.262714711 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 19 == 10
    168257.262715493 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 19 with 1 attributes
    168257.262716189 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.262717339 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 10 (8) (nil) -1431655766
    168257.262718114 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 10 (8) 0xa2c013578e0 8
    168257.263659181 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 128 - 8192, height: 128 - 8192
    168257.263682569 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 19 with 0 attributes
    168257.263689387 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 32 == 11
    168257.263690515 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 32 with 1 attributes
    168257.263691251 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.263692387 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 11 (8) (nil) -1431655766
    168257.263693258 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 11 (8) 0xa2c013578e0 8
    168257.264678166 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 128 - 8192, height: 128 - 8192
    168257.264683301 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 32 with 0 attributes
    168257.264684539 [2006932-2006932] ../src/vabackend.c: 703           nvCreateConfig Unable to determine surface type for VP9/AV1 codec due to no RTFormat specified.
    168257.264688954 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 18 == 8
    168257.264689705 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 18 with 1 attributes
    168257.264690472 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168257.264692002 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 8 (10) (nil) -1431655766
    168257.264693131 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 8 (10) 0xa2c013578e0 8
    168257.265765409 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 144 - 8192, height: 144 - 8192
    168257.265769745 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 18 with 0 attributes
    168257.265774633 [2006932-2006932] ../src/vabackend.c: 572    nvGetConfigAttributes Got here with profile: 21 == 10
    168257.265775419 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 21 with 1 attributes
    168257.265776103 [2006932-2006932] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 256
    168257.265777632 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 10 (10) (nil) -1431655766
    168257.265778679 [2006932-2006932] ../src/vabackend.c:1768 nvQuerySurfaceAttributes with 10 (10) 0xa2c013578e0 8
    168257.266718223 [2006932-2006932] ../src/vabackend.c:1835 nvQuerySurfaceAttributes Returning constraints: width: 128 - 8192, height: 128 - 8192
    168257.266722590 [2006932-2006932] ../src/vabackend.c: 640           nvCreateConfig got profile: 21 with 0 attributes
    168257.266735086 [2006932-2006932] ../src/vabackend.c:2064              nvTerminate Terminating 0xa2c00b7f200
    168257.266779116 [2006932-2006932] ../src/vabackend.c:2078              nvTerminate Now have 0 (0 max) instances
    168268.525625740 [2006932-2007376] ../src/vabackend.c:2154       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 31
    168268.525630921 [2006932-2007376] ../src/vabackend.c:2163       __vaDriverInit_1_0 Now have 0 (0 max) instances
    168268.525634432 [2006932-2007376] ../src/vabackend.c:2189       __vaDriverInit_1_0 Selecting Direct backend
    168268.525655479 [2006932-2007376] ../src/direct/direct-export-buf.c:  92      direct_initExporter We have a drm fd.
    168268.525658203 [2006932-2007376] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
    168268.525660000 [2006932-2007376] ../src/direct/direct-export-buf.c:  95      direct_initExporter drv->drmFd: 26, isNvDrmFd: 0
    168268.525660922 [2006932-2007376] ../src/direct/direct-export-buf.c:  96      direct_initExporter resultCheckModesetParameterFromFd: 1
    168268.525662270 [2006932-2007376] ../src/direct/direct-export-buf.c:  68      direct_initExporter Searching for GPU: 0 0 128
    168268.525710241 [2006932-2007376] ../src/backend-common.c:  31            isNvidiaDrmFd Invalid driver for DRM device: i915
    168268.525717375 [2006932-2007376] ../src/direct/direct-export-buf.c:  68      direct_initExporter Searching for GPU: 0 0 129
    168268.525723788 [2006932-2007376] ../src/direct/direct-export-buf.c:  90      direct_initExporter Found NVIDIA GPU 0 at /dev/dri/renderD129
    168268.525725342 [2006932-2007376] ../src/direct/nv-driver.c: 259            init_nvdriver Initing nvdriver...
    168268.525748307 [2006932-2007376] ../src/direct/nv-driver.c: 277            init_nvdriver NVIDIA kernel driver version: 545.23.08, major version: 545, minor version: 23
    168268.525751317 [2006932-2007376] ../src/direct/nv-driver.c: 284            init_nvdriver Got dev info: 100 1 2 6
    168268.577058173 [2006932-2007376] ../src/vabackend.c: 640           nvCreateConfig got profile: 7 with 1 attributes
    168268.577064032 [2006932-2007376] ../src/vabackend.c: 661           nvCreateConfig got config attrib: 0 0 1
    168269.070782101 [2006932-2007376] ../src/vabackend.c: 987          nvCreateContext creating context with 0 render targets, 0 surfaces, at 1920x1088
    168269.070789324 [2006932-2007376] ../src/vabackend.c:1013          nvCreateContext 0 surfaces have been passed to vaCreateContext, this might cause errors. Setting surface count to 32
    168269.079918456 [2006932-2007380] ../src/vabackend.c: 382          resolveSurfaces [RT] Resolve thread for 0xa2c039a5000 started
[2006932:2007376:0210/185127.505732:ERROR:vaapi_video_decoder.cc(1249)] failed Initialize()ing the frame pool
    168269.081882887 [2006932-2007376] ../src/vabackend.c:1087         nvDestroyContext Destroying context: 2
    168269.081886693 [2006932-2007376] ../src/vabackend.c: 291           destroyContext Signaling resolve thread to exit
    168269.081888225 [2006932-2007376] ../src/vabackend.c: 297           destroyContext Waiting for resolve thread to exit
    168269.081920533 [2006932-2007380] ../src/vabackend.c: 424          resolveSurfaces [RT] Resolve thread for 0xa2c039a5000 exiting
    168269.081943157 [2006932-2007376] ../src/vabackend.c: 299           destroyContext pthread_timedjoin_np finished with 0
    168269.087961603 [2006932-2007376] ../src/vabackend.c:2064              nvTerminate Terminating 0xa2c02f5ae00
    168269.088023564 [2006932-2007376] ../src/vabackend.c:2078              nvTerminate Now have 0 (0 max) instances

And I think it can be used for Chrome/Chromium if we fixed the error?

elFarto commented 5 months ago

Unfortunately I don't think this will work. While it may appear to be initialising the correct device, Chrome doesn't know we're using a different device to the one it's using. When we go to export the DMA-BUF, Chrome will attempt to import it into the EGL/Vulkan context on the Intel card which will fail.

LeviMarvin commented 5 months ago

We have solution to make sure the Chrome using NVIDIA Vulcan, and when I did it, NVIDIA GPU Utilization is float (means the NVIDIA GPU were used). But the video engine still keeps frozen 0%

shelterx commented 5 months ago

And there is an error from Chrome:

[2006932:2007376:0210/185127.505732:ERROR:vaapi_video_decoder.cc(1249)] failed Initialize()ing the frame pool

This is currently affecting Chrome with Nvidia and it's been active for a while. Depending on what flags you run I also got this now which I haven't seen before: [65661:65661:0214/084514.593507:WARNING:vaapi_wrapper.cc(1489)] : Skipping nVidia device named: nvidia-drm

LeviMarvin commented 5 months ago

Yep, so I think there may be a solution to use NVIDIA card. but I have turned to Windows so that I can't test anymore

shelterx commented 4 months ago

Ok, so the previous error is gone but there's another one now, with the latest chromium release I have:

[184461:184765:0226/154948.136812:ERROR:vaapi_wrapper.cc(2322)] : vaCreateContext failed, VA error: resource allocation failed
[184461:184765:0226/154948.136886:ERROR:vaapi_video_decoder.cc(1242)] : failed creating VAContext
lafoletc commented 4 months ago

This is currently affecting Chrome with Nvidia and it's been active for a while. Depending on what flags you run I also got this now which I haven't seen before: [65661:65661:0214/084514.593507:WARNING:vaapi_wrapper.cc(1489)] : Skipping nVidia device named: nvidia-drm

To remove the Nvidia skipping, You need --enable-features=VaapiOnNvidiaGpus

lafoletc commented 4 months ago

how have you removed the following error ?

And there is an error from Chrome:

[2006932:2007376:0210/185127.505732:ERROR:vaapi_video_decoder.cc(1249)] failed Initialize()ing the frame pool
ManuLinares commented 2 months ago

Have any tests been conducted with the latest patch? I haven't achieved any success yet.

wugaosheng123 commented 1 month ago

Can this driver be used on top of chromium.