bootlin / libva-v4l2-request

LibVA implementation for the Linux Video4Linux2 Request API
Other
76 stars 59 forks source link

Not able to use VLC #31

Closed wolfallein closed 4 years ago

wolfallein commented 4 years ago

Hello,

thanks for the work. I'm trying to make the libva work with vlc but without success. I have a Allwinner r16 soc on my device, and I followed the instructions from: https://linux-sunxi.org/Sunxi-Cedrus I compiled kernel 5.7rc5 enabling the cedrus options. I also added 256MB to the CMA, and added cma=256M to my kernel cmdline, but I'm not sure if this is effective because I didn't see decrease on my available RAM, but the dmesg shows:

cpi@clockworkpi:~$ dmesg | grep cma
[    0.000000] cma: Reserved 256 MiB at 0x60000000
[    0.000000] Kernel command line: console=ttyS0,115200n8 earlyprintk no_console_suspend root=/dev/mmcblk0p2 rootfstype=ext4 rootwait init=/sbin/init noinitrd panic=10 cma=256M
[    0.000000] Memory: 761032K/1044976K available (7168K kernel code, 404K rwdata, 1928K rodata, 1024K init, 261K bss, 21800K reserved, 262144K cma-reserved, 258532K highmem)
cpi@clockworkpi:~$ free -h
              total        used        free      shared  buff/cache   available
Mem:          1.0Gi        94Mi       481Mi       8.0Mi       424Mi       876Mi
Swap:            0B          0B          0B

I don't know if I should see less memory with free -h.

With new kernel I have /dev/video0 and /dev/media0 populated

After compiling and installing libva-v4l2-request from master branch (the release-2019.03 didn't compile), I noticed that it installs in the wrong directory (/usr/lib/dri), so I copied v4l2_request_drv_video.so and v4l2_request_drv_video.la to: /usr/lib/arm-linux-gnueabihf/dri/. I'm using Debian Bullseye to get latest mesa driver with lima support, and I have gpu acceleration on mali 400. My X is version 1.20.8, and Xorg.log outputs:

[    29.775] (II) modeset(0): glamor X acceleration enabled on Mali400
[    29.775] (II) modeset(0): glamor initialized
....
[    30.105] (II) modeset(0): [DRI2] Setup complete
[    30.105] (II) modeset(0): [DRI2]   DRI driver: sun4i-drm
[    30.105] (II) modeset(0): [DRI2]   VDPAU driver: sun4i-drm
...
[    30.617] (II) AIGLX: Loaded and initialized sun4i-drm
[    30.617] (II) GLX: Initialized DRI2 GL provider for screen 0

Running vainfo I get:

cpi@clockworkpi:~$ vainfo
error: can't connect to X server!
libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.7.0)
vainfo: Driver version: v4l2-request
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointVLD

and v4l2-request-test works properly. But running VLC inside x11 I get errors. I enabled hardware acceleration and also set X11 video output (XCB), and VA-API video decoder, but to make sure that vlc is trying to use the right method I used the following command line to start vlc:

cvlc --no-audio --x11-display :0 --vout x11 --avcodec-hw vaapi big_buck_bunny_720_stereo.mp4

and I get the following error:

cpi@clockworkpi:~$ cvlc --no-audio --x11-display :0 --vout x11 --avcodec-hw vaapi big_buck_bunny_720_stereo.mp4
VLC media player 3.0.10 Vetinari (revision 3.0.10-0-g7f145afa84)
[00566a80] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[00586b70] main interface error: no suitable interface module
[004ebb98] main libvlc error: interface "globalhotkeys,none" initialization failed
[00586b70] dummy interface: using the dummy interface module...
[b3102538] xcb_x11 vout display error: shared memory allocation error: Function not implemented
[b29016b0] main video output error: video output creation failed
[b2589430] main decoder error: failed to create video output
[b3102838] xcb_x11 vout display error: shared memory allocation error: Function not implemented
[b29191a8] main video output error: video output creation failed
[b2589430] main decoder error: failed to create video output
[b3100898] xcb_x11 vout display error: shared memory allocation error: Function not implemented
[b29191a8] main video output error: video output creation failed
[b2589430] main decoder error: failed to create video output
[h264 @ 0xb258c2c0] get_buffer() failed
[h264 @ 0xb258c2c0] thread_get_buffer() failed
[h264 @ 0xb258c2c0] decode_slice_header error
[h264 @ 0xb258c2c0] no frame!
[b3100898] xcb_x11 vout display error: shared memory allocation error: Function not implemented
[b290ffa8] main video output error: video output creation failed
[b2589430] main decoder error: failed to create video output

My screen starts to flicker and no video is shown. In my Xorg.log I get the many lines like this added:

[  1074.845] (WW) modeset(0): Page flip failed: No such file or directory
[  1074.845] (EE) modeset(0): present flip failed
[  1074.881] (WW) modeset(0): Page flip failed: No such file or directory
[  1074.881] (EE) modeset(0): present flip failed
[  1074.898] (WW) modeset(0): Page flip failed: No such file or directory
[  1074.898] (EE) modeset(0): present flip failed
....

If I change x11 to opengl with:

cvlc --no-audio --x11-display :0 --vout gl --avcodec-hw vaapi big_buck_bunny_720_stereo.mp4

cpi@clockworkpi:~$ cvlc --no-audio --x11-display :0 --vout gl --avcodec-hw vaapi big_buck_bunny_720_stereo.mp4
VLC media player 3.0.10 Vetinari (revision 3.0.10-0-g7f145afa84)
[004aca80] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[004cc8d0] main interface error: no suitable interface module
[00431b98] main libvlc error: interface "globalhotkeys,none" initialization failed
[004cc8d0] dummy interface: using the dummy interface module...
libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
[b3102338] main video output error: video output creation failed
[b2589388] main decoder error: failed to create video output

The video works, the libva seems to be loaded but I have a very high usage of the cpu, and even if I don't set export LIBVA_DRIVER_NAME=v4l2_request the load usage still the same, with the difference that vlc cannot find the v4l2_request and instead look for sun4i-drm_drv_video.so which doesn't exist:

cpi@clockworkpi:~$ cvlc --no-audio --x11-display :0 --vout gl --avcodec-hw vaapi big_buck_bunny_720_stereo.mp4
VLC media player 3.0.10 Vetinari (revision 3.0.10-0-g7f145afa84)
[00572a80] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[005927f8] main interface error: no suitable interface module
[004f7b98] main libvlc error: interface "globalhotkeys,none" initialization failed
[005927f8] dummy interface: using the dummy interface module...
libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/sun4i-drm_drv_video.so
libva info: va_openDriver() returns -1
[b3200d20] glconv_vaapi_x11 gl error: vaInitialize: unknown libva error
libva info: VA-API version 1.7.0
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)

If I set x11 for vout (without export LIBVA_DRIVER_NAME=v4l2_request) I get the same error, and the screen flicker.

So, my problem seems to be in the vlc or xcb.

I tried to compile vlc from source and got the same error from the version in the package manager, but I don't know if I can try to do something to vlc to enable it to work.

If the problem is with x11 or xcb, how can I fix that? What do I have to compile/enable to make it work?

Would libva-v4l2-request work with opengl, or it must be xcb?

To finish, is there another program to try to see if the libva is working with video? Like mplayer, or mpv? If yes, how should I set it?

Thanks a lot!

hob286 commented 4 years ago

"xcb_x11 vout display error: shared memory allocation error: Function not implemented"

You need to add CONFIG_SYSVIPC in your kernel config to support this.

wolfallein commented 4 years ago

Thanks for replying. I wouldn't be able to find out alone.

Today I could work on it and you are right: We need to have CONFIG_SYSVIPC enabled. This wasn't the only problem because v4l2-request needs to be updated with more recent kernels (I'm using 5.7) where they implemented a "decoding mode", so I needed to revert some commits to get v4l2-request working, I found this information on this forum: https://forum.armbian.com/topic/13863-cedrus-on-armbian/ .

Here are my kernel diffs:

diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h
index 1c6ff7d63..601f5647c 100644
--- a/include/media/h264-ctrls.h
+++ b/include/media/h264-ctrls.h
@@ -139,7 +139,7 @@ struct v4l2_ctrl_h264_slice_params {
        __u32 size;

        /* Offset in bytes to the start of slice in the OUTPUT buffer. */
-       __u32 start_byte_offset;
+//     __u32 start_byte_offset;

        /* Offset in bits to slice_data() from the beginning of this slice. */
        __u32 header_bit_size;
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index 05a85517f..bda74fbea 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -77,24 +77,6 @@ static const struct cedrus_control cedrus_controls[] = {
                .codec          = CEDRUS_CODEC_H264,
                .required       = true,
        },
-       {
-               .cfg = {
-                       .id     = V4L2_CID_MPEG_VIDEO_H264_DECODE_MODE,
-                       .max    = V4L2_MPEG_VIDEO_H264_DECODE_MODE_SLICE_BASED,
-                       .def    = V4L2_MPEG_VIDEO_H264_DECODE_MODE_SLICE_BASED,
-               },
-               .codec          = CEDRUS_CODEC_H264,
-               .required       = false,
-       },
-       {
-               .cfg = {
-                       .id     = V4L2_CID_MPEG_VIDEO_H264_START_CODE,
-                       .max    = V4L2_MPEG_VIDEO_H264_START_CODE_NONE,
-                       .def    = V4L2_MPEG_VIDEO_H264_START_CODE_NONE,
-               },
-               .codec          = CEDRUS_CODEC_H264,
-               .required       = false,
-       },
        {
                .cfg = {
                        .id     = V4L2_CID_MPEG_VIDEO_HEVC_SPS,

Running vlc I have:

cpi@clockworkpi :~/src$ cvlc --no-audio --x11-display :0 --vout x11 --avcodec-hw vaapi big_buck_bunny_720_stereo.mp4
VLC media player 3.0.11 Vetinari (revision 3.0.11-0-gdc0c5ced72)
[005402d0] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[00578bd0] dbus interface error: Failed to connect to the D-Bus session daemon: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
[00578bd0] main interface error: no suitable interface module
[004c4b98] main libvlc error: interface "dbus,none" initialization failed
[005635a0] main interface error: no suitable interface module
[004c4b98] main libvlc error: interface "globalhotkeys,none" initialization failed
[005635a0] dummy interface: using the dummy interface module...
error: XDG_RUNTIME_DIR not set in the environment.
error: XDG_RUNTIME_DIR not set in the environment.
[b299aa88] dbus_screensaver inhibit error: cannot connect to session bus: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
libva info: VA-API version 1.8.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0
[b2a893f8] avcodec decoder: Using v4l2-request for hardware decoding
[a9f25ae0] blend blend error: no matching alpha blending routine (chroma: YUVA -> VAOP)
[a9f25ae0] main blend error: blending YUVA to VAOP failed
...

Some errors but the video seems to play correctly and using hardware decoding. CPU cores are at ~20%.

Using software decoding with:

cvlc --no-audio --x11-display :0 --vout x11 --avcodec-hw none big_buck_bunny_720_stereo.mp4

Also gives me CPU cores at around 20%, I don't see an improvement here. When trying 1080p where I should see a difference I get segmentation fault or the video is glitched.

Running FFmpeg now gives me:

time ffmpeg -hwaccel vaapi -hwaccel_device /dev/video0 -hwaccel_output_format vaapi -i 
big_buck_bunny_720_stereo.mp4 -f null /dev/null
...
frame=14315 fps=532 q=-0.0 Lsize=N/A time=00:09:56.47 bitrate=N/A speed=22.2x
...
real    0m27.341s
user    0m28.326s
sys 0m6.613s

I still get this error at the end:

[AVHWFramesContext @ 0xaa746f10] Failed to destroy surface 0x4000000: 6 (invalid VASurfaceID).

The 4 CPU cores are at ~50%

Whereas using software decoding:

time ffmpeg -i big_buck_bunny_720_stereo.mp4 -f null /dev/null
...
frame=14315 fps=222 q=-0.0 Lsize=N/A time=00:09:56.47 bitrate=N/A speed=9.24x
...
real    1m5.084s
user    3m3.382s
sys 0m5.675s

And the 4 CPU cores are at ~90%, a very clear difference :smile:

Again, thanks a lot for your help. I believe that this can help someone else.

BR