search

JeffyCN / meta-rockchip

Yocto BSP layer for the Rockchip SOC boards
Other
114 stars 87 forks source link

How to get the v4l2-m2m decoder working? #34

Closed MastaG closed 1 year ago

MastaG commented 1 year ago

Hi JeffyCN,

I'm struggling a bit for while now to get the v4l2-m2m decoder to work with chromium-ozone-wayland. I've built it with your patchset on top of it. Then I'm running Chromium with: --use-gl=egl --ozone-platform=wayland --in-process-gpu --no-sandbox --gpu-sandbox-start-early --ignore-gpu-blocklist --ignore-gpu-blacklist --enable-accelerated-video-decode --enable-features=VaapiVideoDecoder,VaapiVideoEncoder

When I go to the chrome://gpu patch it says:

Graphics Feature Status
Canvas: Hardware accelerated
Canvas out-of-process rasterization: Disabled
Direct Rendering Display Compositor: Disabled
Compositing: Hardware accelerated
Multiple Raster Threads: Enabled
OpenGL: Enabled
Rasterization: Hardware accelerated
Raw Draw: Disabled
Video Decode: Hardware accelerated
Video Encode: Hardware accelerated
Vulkan: Disabled
WebGL: Hardware accelerated
WebGL2: Hardware accelerated
WebGPU: Disabled
Driver Bug Workarounds
clear_uniforms_before_first_program_use
count_all_in_varyings_packing
enable_webgl_timer_query_extensions
exit_on_context_lost
disabled_extension_GL_KHR_blend_equation_advanced
disabled_extension_GL_KHR_blend_equation_advanced_coherent
Problems Detected
WebGPU has been disabled via blocklist or the command line.
Disabled Features: webgpu
Clear uniforms before first program use on all platforms: [124764](http://crbug.com/124764), [349137](http://crbug.com/349137)
Applied Workarounds: clear_uniforms_before_first_program_use
Mesa drivers in Linux handle varyings without static use incorrectly: [333885](http://crbug.com/333885)
Applied Workarounds: count_all_in_varyings_packing
Disable KHR_blend_equation_advanced until cc shaders are updated: [661715](http://crbug.com/661715)
Applied Workarounds: disable(GL_KHR_blend_equation_advanced), disable(GL_KHR_blend_equation_advanced_coherent)
Expose WebGL's disjoint_timer_query extensions on platforms with site isolation: [808744](http://crbug.com/808744), [870491](http://crbug.com/870491)
Applied Workarounds: enable_webgl_timer_query_extensions
Some drivers can't recover after OUT_OF_MEM and context lost: [893177](http://crbug.com/893177)
Applied Workarounds: exit_on_context_lost
DAWN Info

I do have symlinks created for my video encoder and decoder:

/dev/video-dec -> /dev/video5
/dev/video-dec0 -> /dev/video5
/dev/video-enc -> /dev/video7
/dev/video-enc0 -> /dev/video7

But when I play a video from youtube with h264ify enabled it always seems to select ffmpeg:

render_id: 12
player_id: 32
created: 2022-09-21 09:18:56.014 UTC
origin_url: https://www.youtube.com/
kFrameUrl: https://www.youtube.com/watch?v=mtT_w4gegsY
kFrameTitle: YouTube
url: blob:https://www.youtube.com/d3e14a05-3e01-4ab3-aec3-2ec9db469880
kTextTracks: 
info: Selected video track: [1]
kRendererName: RendererImpl
pipeline_state: kSuspended
duration: 1586.015782
kVideoTracks: [object Object]
kAudioTracks: [object Object]
event: kSuspended
kIsAudioDecryptingDemuxerStream: false
kAudioDecoderName: FFmpegAudioDecoder
kIsPlatformAudioDecoder: false
kIsVideoDecryptingDemuxerStream: false
kVideoDecoderName: FFmpegVideoDecoder
kIsPlatformVideoDecoder: false
dimensions: 1280x720
kResolution: 1280x720
pipeline_buffering_state: [object Object]
video_buffering_state: [object Object]
kVideoPlaybackRoughness: 40.111
kVideoPlaybackFreezing: 0.076659
kFramerate: 25
seek_target: 463.992598
audio_buffering_state: [object Object]

Log:

{
  "properties": {
    "render_id": 12,
    "player_id": 32,
    "created": "2022-09-21 09:18:56.014 UTC",
    "origin_url": "https://www.youtube.com/",
    "kFrameUrl": "https://www.youtube.com/watch?v=mtT_w4gegsY",
    "kFrameTitle": "YouTube",
    "url": "blob:https://www.youtube.com/d3e14a05-3e01-4ab3-aec3-2ec9db469880",
    "kTextTracks": [],
    "info": "Selected video track: [1]",
    "kRendererName": "RendererImpl",
    "pipeline_state": "kSuspended",
    "duration": 1586.015782,
    "kVideoTracks": [
      {
        "alpha mode": "is_opaque",
        "codec": "h264",
        "coded size": "1280x720",
        "color space": "{primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}",
        "encryption scheme": "Unencrypted",
        "has extra data": false,
        "hdr metadata": "unset",
        "natural size": "1280x720",
        "orientation": "0°",
        "profile": "h264 main",
        "visible rect": "0,0 1280x720"
      }
    ],
    "kAudioTracks": [
      {
        "bytes per channel": 2,
        "bytes per frame": 4,
        "channel layout": "STEREO",
        "channels": 2,
        "codec": "aac",
        "codec delay": 0,
        "discard decoder delay": false,
        "encryption scheme": "Unencrypted",
        "has extra data": false,
        "profile": "unknown",
        "sample format": "Signed 16-bit",
        "samples per second": 44100,
        "seek preroll": "0us"
      }
    ],
    "event": "kSuspended",
    "kIsAudioDecryptingDemuxerStream": false,
    "kAudioDecoderName": "FFmpegAudioDecoder",
    "kIsPlatformAudioDecoder": false,
    "kIsVideoDecryptingDemuxerStream": false,
    "kVideoDecoderName": "FFmpegVideoDecoder",
    "kIsPlatformVideoDecoder": false,
    "dimensions": "1280x720",
    "kResolution": "1280x720",
    "pipeline_buffering_state": {
      "for_suspended_start": false,
      "state": "BUFFERING_HAVE_ENOUGH"
    },
    "video_buffering_state": {
      "state": "BUFFERING_HAVE_ENOUGH"
    },
    "kVideoPlaybackRoughness": 40.111,
    "kVideoPlaybackFreezing": 0.076659,
    "kFramerate": 25,
    "seek_target": 463.992598,
    "audio_buffering_state": {
      "state": "BUFFERING_HAVE_ENOUGH"
    }
  },
  "events": [
    {
      "time": 0,
      "key": "created",
      "value": "2022-09-21 09:18:56.014 UTC"
    },
    {
      "time": 10.375,
      "key": "origin_url",
      "value": "https://www.youtube.com/"
    },
    {
      "time": 10.416000008583069,
      "key": "kFrameUrl",
      "value": "https://www.youtube.com/watch?v=mtT_w4gegsY"
    },
    {
      "time": 10.437000036239624,
      "key": "kFrameTitle",
      "value": "YouTube"
    },
    {
      "time": 10.946000039577484,
      "key": "url",
      "value": "blob:https://www.youtube.com/d3e14a05-3e01-4ab3-aec3-2ec9db469880"
    },
    {
      "time": 10.983000040054321,
      "key": "kTextTracks",
      "value": []
    },
    {
      "time": 11.117000043392181,
      "key": "info",
      "value": "ChunkDemuxer"
    },
    {
      "time": 11.189999997615814,
      "key": "kRendererName",
      "value": "RendererImpl"
    },
    {
      "time": 11.592999994754791,
      "key": "pipeline_state",
      "value": "kStarting"
    },
    {
      "time": 233.43700003623962,
      "key": "duration",
      "value": 1586
    },
    {
      "time": 2241.9050000309944,
      "key": "kVideoTracks",
      "value": [
        {
          "alpha mode": "is_opaque",
          "codec": "h264",
          "coded size": "1280x720",
          "color space": "{primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}",
          "encryption scheme": "Unencrypted",
          "has extra data": false,
          "hdr metadata": "unset",
          "natural size": "1280x720",
          "orientation": "0°",
          "profile": "h264 main",
          "visible rect": "0,0 1280x720"
        }
      ]
    },
    {
      "time": 2246.5139999985695,
      "key": "kAudioTracks",
      "value": [
        {
          "bytes per channel": 2,
          "bytes per frame": 4,
          "channel layout": "STEREO",
          "channels": 2,
          "codec": "aac",
          "codec delay": 0,
          "discard decoder delay": false,
          "encryption scheme": "Unencrypted",
          "has extra data": false,
          "profile": "unknown",
          "sample format": "Signed 16-bit",
          "samples per second": 44100,
          "seek preroll": "0us"
        }
      ]
    },
    {
      "time": 2282.538000047207,
      "key": "info",
      "value": "Effective playback rate changed from 0 to 1"
    },
    {
      "time": 2283.388000011444,
      "key": "event",
      "value": "kPlay"
    },
    {
      "time": 2304.526000022888,
      "key": "kIsAudioDecryptingDemuxerStream",
      "value": false
    },
    {
      "time": 2304.550000011921,
      "key": "kAudioDecoderName",
      "value": "FFmpegAudioDecoder"
    },
    {
      "time": 2304.5740000009537,
      "key": "kIsPlatformAudioDecoder",
      "value": false
    },
    {
      "time": 2304.6930000185966,
      "key": "info",
      "value": "Selected FFmpegAudioDecoder for audio decoding, config: codec: aac, profile: unknown, bytes_per_channel: 2, channel_layout: STEREO, channels: 2, samples_per_second: 44100, sample_format: Signed 16-bit, bytes_per_frame: 4, seek_preroll: 0us, codec_delay: 0, has extra data: false, encryption scheme: Unencrypted, discard decoder delay: false, target_output_channel_layout: STEREO, target_output_sample_format: Unknown sample format, has aac extra data: true"
    },
    {
      "time": 2309.1830000281334,
      "key": "info",
      "value": "Failed to initialize DecryptingVideoDecoder"
    },
    {
      "time": 2309.407000005245,
      "key": "info",
      "value": "Failed to initialize VDAVideoDecoder"
    },
    {
      "time": 2309.587999999523,
      "key": "info",
      "value": "Failed to initialize VpxVideoDecoder"
    },
    {
      "time": 2309.7549999952316,
      "key": "info",
      "value": "Failed to initialize Dav1dVideoDecoder"
    },
    {
      "time": 2339.0470000505447,
      "key": "kIsVideoDecryptingDemuxerStream",
      "value": false
    },
    {
      "time": 2339.069000005722,
      "key": "kVideoDecoderName",
      "value": "FFmpegVideoDecoder"
    },
    {
      "time": 2339.0830000042915,
      "key": "kIsPlatformVideoDecoder",
      "value": false
    },
    {
      "time": 2339.240999996662,
      "key": "info",
      "value": "Selected FFmpegVideoDecoder for video decoding, config: codec: h264, profile: h264 main, level: not available, alpha_mode: is_opaque, coded size: [1280,720], visible rect: [0,0,1280,720], natural size: [1280,720], has extra data: false, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}"
    },
    {
      "time": 2347.6240000128746,
      "key": "pipeline_state",
      "value": "kPlaying"
    },
    {
      "time": 3007.867000043392,
      "key": "dimensions",
      "value": "1280x720"
    },
    {
      "time": 3007.888000011444,
      "key": "kResolution",
      "value": "1280x720"
    },
    {
      "time": 2277.5329999923706,
      "key": "duration",
      "value": 1586.015782
    },
    {
      "time": 3181.6579999923706,
      "key": "pipeline_buffering_state",
      "value": {
        "for_suspended_start": false,
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 4407.51700001955,
      "key": "video_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 16307.198000013828,
      "key": "info",
      "value": "Selected video track: []"
    },
    {
      "time": 16313.341000020504,
      "key": "pipeline_buffering_state",
      "value": {
        "for_suspended_start": false,
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 36876.04199999571,
      "key": "info",
      "value": "Selected video track: [1]"
    },
    {
      "time": 36882.905000030994,
      "key": "video_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_NOTHING"
      }
    },
    {
      "time": 38010.06700003147,
      "key": "pipeline_buffering_state",
      "value": {
        "for_suspended_start": false,
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 74910.87800002098,
      "key": "event",
      "value": "kPause"
    },
    {
      "time": 74914.78299999237,
      "key": "kVideoPlaybackRoughness",
      "value": 40.111
    },
    {
      "time": 74926.71700000763,
      "key": "kVideoPlaybackFreezing",
      "value": 0.076659
    },
    {
      "time": 74926.75300002098,
      "key": "kFramerate",
      "value": 25
    },
    {
      "time": 75144.79199999571,
      "key": "seek_target",
      "value": 418.861019
    },
    {
      "time": 75145.27100002766,
      "key": "pipeline_state",
      "value": "kSeeking"
    },
    {
      "time": 75253.97000002861,
      "key": "event",
      "value": "kPlay"
    },
    {
      "time": 75807.1360000372,
      "key": "pipeline_state",
      "value": "kPlaying"
    },
    {
      "time": 75965.83200001717,
      "key": "audio_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 77411.65600001812,
      "key": "video_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 77412.45700001717,
      "key": "pipeline_buffering_state",
      "value": {
        "for_suspended_start": false,
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 79325.08200001717,
      "key": "event",
      "value": "kPause"
    },
    {
      "time": 79405.79300004244,
      "key": "seek_target",
      "value": 463.992598
    },
    {
      "time": 79406.42200005054,
      "key": "pipeline_state",
      "value": "kSeeking"
    },
    {
      "time": 79502.63700002432,
      "key": "event",
      "value": "kPlay"
    },
    {
      "time": 79409.49099999666,
      "key": "video_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_NOTHING"
      }
    },
    {
      "time": 80003.09700000286,
      "key": "pipeline_state",
      "value": "kPlaying"
    },
    {
      "time": 80119.97900003195,
      "key": "audio_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 81976.74099999666,
      "key": "video_buffering_state",
      "value": {
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 82020.19400000572,
      "key": "pipeline_buffering_state",
      "value": {
        "for_suspended_start": false,
        "state": "BUFFERING_HAVE_ENOUGH"
      }
    },
    {
      "time": 105037.28700000048,
      "key": "event",
      "value": "kPause"
    },
    {
      "time": 123379.98600000143,
      "key": "pipeline_state",
      "value": "kSuspending"
    },
    {
      "time": 123409.16000002623,
      "key": "pipeline_state",
      "value": "kSuspended"
    },
    {
      "time": 123409.35199999809,
      "key": "event",
      "value": "kSuspended"
    }
  ]
}

Any ideas? Kodi is working fine with FFMPeg's v4l2m2m decoder.

JeffyCN commented 1 year ago

the chromium's v4l2vda has some special requirements, not for all v4l2mem devices.

the official one should be tegra.

for rockchip, i'm using a v4l2 plugin to wrap mpp apis to adapt to chromium's requirements: https://github.com/JeffyCN/libv4l-rkmpp#faq

the patches here, are just to enable the v4l2vda, you might need to enable debugging logs and make your own patch to adapt your driver(or in chromium, or make another v4l2 wrap plugin)