How can I enable the VAAPI for RGB processing

[mzwang828]

Hi,

I'm running a kinect 2 on i5-4570T and GTX-1050 and I have very low fps for RGB images. I did the Performance test and here is the output.

administrator@CPR-J100-0136:~/libfreenect2/build$ ./bin/Protonect -noviewer cuda ./bin/Protonect: /usr/local/cuda-8.0/lib64/libOpenCL.so.1: no version information available (required by /home/administrator/libfreenect2/build/lib/libfreenect2.so.0.2) ./bin/Protonect: /usr/local/cuda-8.0/lib64/libOpenCL.so.1: no version information available (required by /home/administrator/libfreenect2/build/lib/libfreenect2.so.0.2) Version: 0.2.0 Environment variables: LOGFILE= Usage: ./bin/Protonect [-gpu=] [gl | cl | clkde | cuda | cudakde | cpu] [] [-noviewer] [-norgb | -nodepth] [-help] [-version] [-frames ] To pause and unpause: pkill -USR1 Protonect [Info] [CudaDepthPacketProcessorImpl] device 0: GeForce GTX 1050 Ti @ 1468MHz Memory 4038MB [Info] [CudaDepthPacketProcessorImpl] selected device 0 [Error] [VaapiRgbPacketProcessorImpl] vaDisplayIsValid(display) failed [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 11 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @4:5 with serial 502777741942 [Info] [Freenect2Impl] found 1 devices [Info] [Freenect2DeviceImpl] opening... [Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 2016384 ir: 608*33792 [Info] [Freenect2DeviceImpl] opened [Info] [Freenect2DeviceImpl] starting... [Debug] [Freenect2DeviceImpl] status 0x090000: 9729 [Debug] [Freenect2DeviceImpl] status 0x090000: 9731 [Info] [Freenect2DeviceImpl] submitting rgb transfers... [Info] [Freenect2DeviceImpl] submitting depth transfers... [Info] [Freenect2DeviceImpl] started device serial: 502777741942 device firmware: 4.0.3911.0 [Debug] [DepthPacketStreamParser] not all subsequences received 0 [Debug] [DepthPacketStreamParser] not all subsequences received 512 [Info] [CudaDepthPacketProcessor] avg. time: 1.40146ms -> ~713.54Hz [Info] [TurboJpegRgbPacketProcessor] avg. time: 21.8688ms -> ~45.7272Hz The viewer is turned off. Received 100 frames. Ctrl-C to stop. [Info] [CudaDepthPacketProcessor] avg. time: 1.38034ms -> ~724.461Hz [Info] [TurboJpegRgbPacketProcessor] avg. time: 22.0983ms -> ~45.2524Hz The viewer is turned off. Received 200 frames. Ctrl-C to stop. [Info] [CudaDepthPacketProcessor] avg. time: 1.382ms -> ~723.587Hz [Info] [TurboJpegRgbPacketProcessor] avg. time: 22.1543ms -> ~45.1379Hz The viewer is turned off. Received 300 frames. Ctrl-C to stop. [Info] [CudaDepthPacketProcessor] avg. time: 1.36717ms -> ~731.439Hz [Info] [TurboJpegRgbPacketProcessor] avg. time: 21.9801ms -> ~45.4957

I can see that the RGB was processed by TurboJpeg, but from the cmake output, I can see that VAAPI is installed:

-- Feature list:
--   CUDA    yes
--   CXX11    yes
--   Examples    yes
--   OpenCL    yes
--   OpenGL    yes
--   OpenNI2    yes
--   TegraJPEG    no
--   Threading    std::thread
--   TurboJPEG    yes
--   VAAPI    yes
--   VideoToolbox    no (Apple only)
-- Configuring done

My question is how can I enable the VAAPI to accelerate the RGB image?

Thank you.

[xlz]

[Error] [VaapiRgbPacketProcessorImpl] vaDisplayIsValid(display) failed

This says something about driver issue.

See this page https://wiki.archlinux.org/index.php/Hardware_video_acceleration#Verification for more info on how to diagnose this. You will use vainfo to verify drivers. Probably also LIBVA_DRIVER_NAME to configure the drivers.

[mzwang828]

Thank you xlz!

I tried to verify drivers using vainfo, and I get this output

error: can't connect to X server! error: can't open DRM connection! error: failed to initialize display Aborted (core dumped)

I couldn't find anything in google regarding the can't open DRM connection error. I also tried to run vainfo in root, which gives me

error: XDG_RUNTIME_DIR not set in the environment. error: can't connect to X server! libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns -1 libva error: va_getDriverName() failed with unknown libva error,driver_name=(null) vaInitialize failed with error code -1 (unknown libva error),exit

still couldn't find anything on libva info: va_getDriverName() returns -1 either......

Do you have any idea what is wrong here?

[mzwang828]

tried using the LIBVA_DRIVER_NAME variable:

administrator@CPR-J100-0136:~$ sudo LIBVA_DRIVER_NAME=i965 vainfo [sudo] password for administrator: error: XDG_RUNTIME_DIR not set in the environment. error: can't connect to X server! libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns -1 libva info: User requested driver 'i965' libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 vainfo: intel_driver.c:100: intel_driver_init: Assertion `(((struct drm_state )(ctx)->drm_state)->auth_type == (VA_DRM_AUTH_DRI1)) || (((struct drm_state )(ctx)->drm_state)->auth_type == (VA_DRM_AUTH_DRI2)) || (((struct drm_state *)(ctx)->drm_state)->auth_type == (VA_DRM_AUTH_CUSTOM))' failed.

[mzwang828]

I need to make a correction... I just check the CPU I have, it's actually a Celeron G1840, although the tech spec of the robot told me it has an i5. I did a search on G1840 and it should still support VAAPI. Also, I'm running a Ubuntu 14.04 server, could this be an issue?

[xlz]

You are using Nvidia for your desktop graphics. This setup is brittle. Intel for desktop graphics and NVidia for compute should works better.

After a clean setup doing this

(Nvidia/Intel dual GPUs) After apt-get install cuda, use sudo prime-select intel to use Intel GPU for desktop.

Should make it work. But it's more complicated how to modify existing graphics drivers. I can't tell you how.

[mzwang828]

Thank you! That must be the issue. I uninstalled everything relative to the nvidia drive and now the vainfo looks much better.

administrator@CPR-J100-0136:~$ vainfo error: can't connect to X server! libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.39 (libva 1.3.0) vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Desktop - 1.7.0 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice unknown profile : VAEntrypointVLD unknown profile : VAEntrypointEncSlice unknown profile : VAEntrypointVLD unknown profile : VAEntrypointEncSlice VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD

I got one more question before trying to installing nvidia driver and cuda back. As shown in the vainfo, I only see VAProfileJPEGBaseline : VAEntrypointVLD but no VAProfileJPEGBaseline : VAEntrypointEncPicture. Will this be a problem? Do I need the encoding ability here?

[xlz]

This is alright. Only decoding is needed.

[mzwang828]

Hi xlz, I thought I was good but there is still problem.

After uninstalled Nvidia driver, I got a good vainfo output. I then reinstalled nvidia driver and cuda follow this instruction:

https://devtalk.nvidia.com/default/topic/842208/does-cuda-depend-on-nvidia-graphics-driver-/ https://askubuntu.com/questions/830983/how-to-winstall-nvidia-drivers-to-use-cuda-without-also-installing-x11

My vainfo become bad again, but this time sudo vainfo is good.

administrator@CPR-J100-0136:~/libfreenect2/build$ vainfo error: can't connect to X server! error: can't open DRM connection! error: failed to initialize display Aborted (core dumped)

administrator@CPR-J100-0136:~/libfreenect2/build$ sudo vainfo error: XDG_RUNTIME_DIR not set in the environment. error: can't connect to X server! libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.39 (libva 1.3.0) vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Desktop - 1.7.0 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice unknown profile : VAEntrypointVLD unknown profile : VAEntrypointEncSlice unknown profile : VAEntrypointVLD unknown profile : VAEntrypointEncSlice VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD

When I run the performance test, vaapi is not enabled. Do you have any suggestion? I can reinstall ubuntu to get a fresh new environment, but that would be the worst case.

[mzwang828]

Finally, the problem is solved. So after the above operation, I installed another driver libva-intel-vaapi-driver I add myself to the video group and then vainfo gives right output and perform test is using VAAPI. For future reference, when using Ubuntu server with dual cards, one can refer to the above link to install Nvidia driver and CUDA without installing unnecessary dependencies and the nvidia card will only be used for computing.

Thanks again!

[xlz]

Dual graphics setup can be complicated indeed. I was always in the video group before and the vaapi-driver package was always pulled in by some xorg-intel package but these things also depend on how your distro manage things. It has to take some tinkering.