search

Miouyouyou / FFmpeg-V4L2-Request

Script to download and compile FFMPEG with V4L2 Request API support.
8 stars 1 forks source link

Patches don't apply to current master #1

Closed rubenvb closed 4 years ago

rubenvb commented 4 years ago

I'm trying to get V4L2_request support built into ffmpeg for testing on my rk3399 NanoPi M4. My last attempt to get something failed when your script failed to apply patches to current FFmpeg master.

End goal is to have a Kodi build that uses v4l2 (request api as I think that's the only thing being worked on upstream for rk3399). Below is the backstory of what I've tried. I have some kernel patches applied, namely these here: https://github.com/rubenvb/PKGBUILDs/commit/26c17bfded37daf8d0676e61e571920ae92c721e

If I run v4l2-ctl --all I get promising output from my patched kernel build:

Driver Info:
        Driver name      : rkvdec
        Card type        : rkvdec
        Bus info         : platform:rkvdec
        Driver version   : 5.6.2
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 48/48
        Pixel Format      : 'NV12' (Y/CbCr 4:2:0)
        Field             : None
        Number of planes  : 1
        Flags             : 
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 48
           Size Image     : 4608
Format Video Output Multiplanar:
        Width/Height      : 48/48
        Pixel Format      : 'S264' (H.264 Parsed Slice Data)
        Field             : None
        Number of planes  : 1
        Flags             : 
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 0
           Size Image     : 4608

Codec Controls

    h264_sequence_parameter_set 0x00990ce8 (unknown): type=110 flags=has-payload
     h264_picture_parameter_set 0x00990ce9 (unknown): type=111 flags=has-payload
            h264_scaling_matrix 0x00990cea (unknown): type=112 flags=has-payload
          h264_slice_parameters 0x00990ceb (unknown): type=113 flags=has-payload
         h264_decode_parameters 0x00990cec (unknown): type=114 flags=has-payload
               h264_decode_mode 0x00990ced (menu)   : min=1 max=1 default=1 value=1
                                1: Frame-Based
                h264_start_code 0x00990cee (menu)   : min=1 max=1 default=1 value=1
                                1: Annex B Start Code

I have succesfully build FFmpeg (4.0.4, Kodi Leia version) with the LibreElec patches from here: https://github.com/LibreELEC/LibreELEC.tv/tree/libreelec-9.2/packages/multimedia/ffmpeg/patches/v4l2-request-api

But I don't know how to actually use the v4l2 request hwaccel. ffmpeg -hwaccels only lists 

vdpau
vaapi
drm

which seems to contradict the inital output from FFmpeg's configure:

[...]
External libraries providing hardware acceleration:
v4l2_m2m                v4l2_request            vaapi                   vdpau
Enabled decoders:
[...] h263_v4l2m2m [...] h264_v4l2m2m [...] hevc_v4l2m2m [...] mpeg1_v4l2m2m [...] mpeg2_v4l2m2m [...] mpeg4_v4l2m2m [...] vc1_v4l2m2m [...] vp8_v4l2m2m [...] vp9_v4l2m2m [...]
[...]
Enabled hwaccels:
h263_vaapi              h264_vaapi              hevc_v4l2request        hevc_vdpau              mpeg1_vdpau             mpeg2_vaapi             mpeg4_vaapi             vc1_vaapi               vp8_vaapi               wmv3_vaapi
h264_v4l2request        h264_vdpau              hevc_vaapi              mjpeg_vaapi             mpeg2_v4l2request       mpeg2_vdpau             mpeg4_vdpau             vc1_vdpau               vp9_vaapi               wmv3_vdpau
[...]
Enabled indevs:
alsa                    fbdev                   kmsgrab                 lavfi                   oss                     sndio                   v4l2                    xcbgrab
Enabled outdevs:
alsa                    fbdev                   oss                     sdl2                    sndio                   v4l2                    xv

I don't know what I can try next, so I made this issue as you at least seem to know what you're doing :). It might just be that the kernel patches I found are insufficient, I already contacted the author about the completeness of these patches but he hasn't responded yet.

Thanks for any help you may be able to provide.

rubenvb commented 4 years ago

I have also spotted a difference between my kernel's h264-ctrls.h and the one added in the patches. Replacing it with the one from my kernel build makes the build fail on undefined V4L2_PIX_FMT_H264_SLICE_RAW. So I removed the _RAW to match this kernel change: https://lkml.org/lkml/2019/8/12/965 This seems a cosmetic change at this level in any case.

Miouyouyou commented 4 years ago

Hmm, are you using a 5.6.2 kernel ?

Anyway, most of my tests were done using mpv compiled against FFMPEG. Do you have multiple versions of FFMPEG installed on your system ? In that case, could you paste the output of

ldd $(which ffmpeg)

I'm used to execute mpv like this :

./mpv --vo=gpu --gpu-context=drm --hwdec=drm video_file.mkv -v --msg-level=ffmpeg=trace

Then I quit the player as soon as possible and look at the logs.

The player is launched in console mode (chvt 1), since it uses the DRM output directly.

I reckon that testing the driver is kind of a pain, and its developers seem to be oblivious to this (or maybe they have something right now... I don't know who's currently actively developing the client part actually).

rubenvb commented 4 years ago

Yes, linux 5.6.2 with hopefully the correct v4l2 patches applied. I thought that would give me the least headache when applying these bleeding-edge patches, and at least the kernel wasn't all that hard to get build, seemingly correctly providing the new rkvdev device which at least spits back some strings containing h264.

Here's the ldd ./ffmpeg I built and tried and failed:

$ ldd ./ffmpeg
        linux-vdso.so.1 (0x0000ffff9e7b8000)
        libXv.so.1 => /usr/lib/libXv.so.1 (0x0000ffff9cfe3000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0x0000ffff9ce8f000)
        libXext.so.6 => /usr/lib/libXext.so.6 (0x0000ffff9ce6b000)
        libm.so.6 => /usr/lib/libm.so.6 (0x0000ffff9cdbf000)
        libdrm.so.2 => /usr/lib/libdrm.so.2 (0x0000ffff9cd9b000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0x0000ffff9cd62000)
        libxcb-shm.so.0 => /usr/lib/libxcb-shm.so.0 (0x0000ffff9cd4e000)
        libxcb-shape.so.0 => /usr/lib/libxcb-shape.so.0 (0x0000ffff9cd3a000)
        libxcb-xfixes.so.0 => /usr/lib/libxcb-xfixes.so.0 (0x0000ffff9cd21000)
        libasound.so.2 => /usr/lib/libasound.so.2 (0x0000ffff9cc2b000)
        libSDL2-2.0.so.0 => /usr/lib/libSDL2-2.0.so.0 (0x0000ffff9cabd000)
        libsndio.so.7.0 => /usr/lib/libsndio.so.7.0 (0x0000ffff9ca9a000)
        libva.so.2 => /usr/lib/libva.so.2 (0x0000ffff9ca62000)
        libbz2.so.1.0 => /usr/lib/libbz2.so.1.0 (0x0000ffff9ca41000)
        libz.so.1 => /usr/lib/libz.so.1 (0x0000ffff9ca1a000)
        liblzma.so.5 => /usr/lib/liblzma.so.5 (0x0000ffff9c9e4000)
        libudev.so.1 => /usr/lib/libudev.so.1 (0x0000ffff9c99e000)
        libva-drm.so.2 => /usr/lib/libva-drm.so.2 (0x0000ffff9c98b000)
        libva-x11.so.2 => /usr/lib/libva-x11.so.2 (0x0000ffff9c975000)
        libvdpau.so.1 => /usr/lib/libvdpau.so.1 (0x0000ffff9c961000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x0000ffff9c931000)
        libc.so.6 => /usr/lib/libc.so.6 (0x0000ffff9c7c4000)
        /lib/ld-linux-aarch64.so.1 => /usr/lib/ld-linux-aarch64.so.1 (0x0000ffff9e78a000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x0000ffff9c7b0000)
        libXau.so.6 => /usr/lib/libXau.so.6 (0x0000ffff9c79c000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x0000ffff9c786000)
        librt.so.1 => /usr/lib/librt.so.1 (0x0000ffff9c76e000)
        libbsd.so.0 => /usr/lib/libbsd.so.0 (0x0000ffff9c747000)
        libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0x0000ffff9c731000)

The output of v4l2-compliance is also quite good:

Compliance test for rkvdec device /dev/video1:

Driver Info:
        Driver name      : rkvdec
        Card type        : rkvdec
        Bus info         : platform:rkvdec
        Driver version   : 5.6.2
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : rkvdec
        Model            : rkvdec
        Serial           : 
        Bus info         : platform:rkvdec
        Media version    : 5.6.2
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.6.2
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : rkvdec-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'rkvdec-proc': Data, Enabled, Immutable

Required ioctls:
        test MC information (see 'Media Driver Info' above): OK
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second /dev/video1 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
        test VIDIOC_QUERYCTRL: OK
        test VIDIOC_G/S_CTRL: OK
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 8 Private Controls: 0

Format ioctls:
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK (Not Supported)
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK (Not Supported)
        test Composing: OK (Not Supported)
        test Scaling: OK

Codec ioctls:
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK

Total for rkvdec device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0

I have noticed the /dev/video{0,1,2} devices I have (rockchip-rga: seemingly only scaling/rotation/... and JPEG, hantro-vpu: seemingly only MPEG2, and the new rkvdec: should hopefully provide h264 decoding) aren't assigned the same number each boot.

I'll try to build a replacement system ffmpeg package so I can't get confused anymore between my local builds and system libraries. But I also built a static ffmpeg just to avoid this kind of mixing, same result. ./ffmpeg -hwaccels also does not print the v4l2_request hwaccel even though configure output listed it.

FYI I'm running ArchLinuxARM, hence the recent kernel (I took their linux-aarch64-5.6-rc package as a basis for my kernel). They also have a great distcc cross-compile setup so things happen a bit faster than usual.

The issue title doesn't really correspond to what we're discussing though, sorry for that.

Miouyouyou commented 4 years ago

Hmm, could you try to compile "mpv" against this version of FFMPEG libraries ?

Also :

I haven't checked on ffmpeg -hwaccels work, but maybe the patches don't add the right hooks to display themselves in the list.

Miouyouyou commented 4 years ago

Ok, I tried recompiling a version of FFMPEG with v4l2 request API enabled, but the same thing happened... -hwaccels don't return v4l2_request but don't return xvmc too, so that might be a red herring... I guess that it's going to end with some horrendous printf debugging to understand which functions are called, and why the v4l2 request API is not used...

I'll open a new thread for this.