search

TechNexion / tn-imx-yocto-manifest

TechNexion i.MX Yocto manifests
25 stars 11 forks source link

GPU hardware acceleration #11

Closed VaporMontey closed 2 years ago

VaporMontey commented 3 years ago

Hi,

I've built the image of Yocto 2.5 for my PICO-PI-IMX8MQ but I think I don't have the GPU hw acceleration enabled. When playing a 4K video (on stream) the CPU usage goes to 130%-150% using, in my opinion, software rendering. Is there an option or something I'm missing for using GPU rendering (Hardware acceleration)?

How I created the image:

$: DISTRO=fsl-imx-wayland MACHINE=pico-imx8mq source edm-setup-release.sh -b build-wayland-imx8mq $: bitbake fsl-image-qt5-validation-imx

The following messages from chromium chrome://gpu page:

Graphics_ Feature Status Canvas: Hardware accelerated Flash: Hardware accelerated Flash Stage3D: Hardware accelerated Flash Stage3D Baseline profile: Hardware accelerated Compositing: Hardware accelerated Multiple Raster Threads: Enabled Native GpuMemoryBuffers: Software only. Hardware acceleration disabled Out-of-process Rasterization: Disabled Hardware Protected Video Decode: Unavailable Rasterization: Software only. Hardware acceleration disabled Skia Deferred Display List: Disabled Skia Renderer: Disabled Surface Synchronization: Enabled Video Decode: Unavailable Viz Service Display Compositor: Disabled WebGL: Hardware accelerated WebGL2: Hardware accelerated Driver Bug Workarounds clear_uniforms_before_first_program_use disable_framebuffer_cmaa scalarize_vec_and_mat_constructor_args disabled_extension_GL_KHR_blend_equation_advanced disabled_extension_GL_KHR_blend_equation_advanced_coherent Problems Detected Accelerated video decode is unavailable on Linux: 137247 Disabled Features: accelerated_video_decode Protected video decoding with swap chain is for Windows and Intel only Disabled Features: protected_video_decode Clear uniforms before first program use on all platforms: 124764, 349137 Applied Workarounds: clear_uniforms_before_first_program_use Always rewrite vec/mat constructors to be consistent: 398694 Applied Workarounds: scalarize_vec_and_mat_constructor_args Use GL_INTEL_framebuffer_CMAA on ChromeOS: 535198 Applied Workarounds: disable_framebuffer_cmaa Disable KHR_blend_equation_advanced until cc shaders are updated: 661715 Applied Workarounds: disable(GL_KHR_blend_equation_advanced), disable(GL_KHR_blend_equation_advanced_coherent) Native GpuMemoryBuffers have been disabled, either via about:flags or command line. Disabled Features: native_gpu_memory_buffers Viz service display compositor is not enabled by default. Disabled Features: viz_display_compositor Skia renderer is not used by default. Disabled Features: skia_renderer Skia deferred display list is not used by default. Disabled Features: skia_deferred_display_list Version Information Data exported 2020-11-05T15:59:44.305Z Chrome version Chrome/71.0.3545.0 Operating system Linux 4.14.98-0cac981fa0cc97ca990d0d98d15f598423a703e4+g0cac981 Software rendering list URL https://chromium.googlesource.com/chromium/src/+/9e15e31d9f39013dda04cf6b9dec7f61ad40204f/gpu/config/software_rendering_list.json Driver bug list URL https://chromium.googlesource.com/chromium/src/+/9e15e31d9f39013dda04cf6b9dec7f61ad40204f/gpu/config/gpu_driver_bug_list.json ANGLE commit id unknown hash 2D graphics backend Skia/71 0c3dc59fc1c61e6f54cf3d0100e54ee0d20ec13d- Command Line /usr/bin/chromium --use-gl=egl --ozone-platform=wayland --in-process-gpu --gpu-preferences=KAAAAAAAAACiAAAAAQAAAAAAAAAAAGAAAAAAAAEAAAAIAAAAAAAAAGgAAAAMAAAAYAAAAAAAAABoAAAAAAAAAHAAAAAAAAAAeAAAAAAAAACAAAAAAAAAAIgAAAAAAAAAkAAAAAAAAACYAAAAAAAAAKAAAAAAAAAAqAAAAAAAAACwAAAAAAAAALgAAAAAAAAAEAAAAAAAAAAAAAAACgAAABAAAAAAAAAAAAAAAAsAAAAQAAAAAAAAAAAAAAAMAAAAEAAAAAAAAAAAAAAADwAAABAAAAAAAAAAAQAAAAoAAAAQAAAAAAAAAAEAAAALAAAAEAAAAAAAAAABAAAADAAAABAAAAAAAAAAAQAAAA8AAAAQAAAAAAAAAAQAAAAKAAAAEAAAAAAAAAAEAAAACwAAABAAAAAAAAAABAAAAAwAAAAQAAAAAAAAAAQAAAAPAAAA --in-process-gpu --no-sandbox --ozone-platform=wayland --use-gl=egl --flag-switches-begin --flag-switches-end --gpu-preferences=KAAAAAAAAACiAAAAAQAAAAAAAAAAAGAAAAAAAAAAAAAIAAAAAAAAAGgAAAAMAAAAYAAAAAAAAABoAAAAAAAAAHAAAAAAAAAAeAAAAAAAAACAAAAAAAAAAIgAAAAAAAAAkAAAAAAAAACYAAAAAAAAAKAAAAAAAAAAqAAAAAAAAACwAAAAAAAAALgAAAAAAAAAEAAAAAAAAAAAAAAACgAAABAAAAAAAAAAAAAAAAsAAAAQAAAAAAAAAAAAAAAMAAAAEAAAAAAAAAAAAAAADwAAABAAAAAAAAAAAQAAAAoAAAAQAAAAAAAAAAEAAAALAAAAEAAAAAAAAAABAAAADAAAABAAAAAAAAAAAQAAAA8AAAAQAAAAAAAAAAQAAAAKAAAAEAAAAAAAAAAEAAAACwAAABAAAAAAAAAABAAAAAwAAAAQAAAAAAAAAAQAAAAPAAAA --use-gl=egl Driver Information Initialization time 1 In-process GPU true Passthrough Command Decoder false Sandboxed false GPU0 VENDOR = 0x0000 [Vivante Corporation], DEVICE= 0x0000 [Vivante GC7000L] ACTIVE Optimus false AMD switchable false Driver vendor
Driver version 6.2.4.p4.190076 Driver date GPU CUDA compute capability major version 0 Pixel shader version 3.10 Vertex shader version 3.10 Max. MSAA samples 4 Machine model name
Machine model version
GL_VENDOR Vivante Corporation GL_RENDERER Vivante GC7000L GL_VERSION OpenGL ES 3.1 V6.2.4.p4.190076 GL_EXTENSIONS GL_OES_vertex_type_10_10_10_2 GL_OES_vertex_half_float GL_OES_element_index_uint GL_OES_mapbuffer GL_OES_vertex_array_object GL_OES_compressed_ETC1_RGB8_texture GL_OES_compressed_paletted_texture GL_OES_texture_npot GL_OES_rgb8_rgba8 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_depth24 GL_OES_depth32 GL_OES_packed_depth_stencil GL_OES_fbo_render_mipmap GL_OES_get_program_binary GL_OES_fragment_precision_high GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_sync GL_OES_texture_stencil8 GL_OES_shader_image_atomic GL_OES_texture_storage_multisample_2d_array GL_OES_required_internalformat GL_OES_surfaceless_context GL_OES_copy_image GL_OES_draw_buffers_indexed GL_OES_texture_border_clamp GL_OES_texture_buffer GL_OES_texture_cube_map_array GL_OES_draw_elements_base_vertex GL_OES_texture_half_float GL_OES_texture_float GL_KHR_blend_equation_advanced GL_KHR_debug GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_texture_type_2_10_10_10_REV GL_EXT_texture_filter_anisotropic GL_EXT_texture_compression_dxt1 GL_EXT_texture_format_BGRA8888 GL_EXT_texture_compression_s3tc GL_EXT_read_format_bgra GL_EXT_multi_draw_arrays GL_EXT_frag_depth GL_EXT_discard_framebuffer GL_EXT_blend_minmax GL_EXT_multisampled_render_to_texture GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float GL_EXT_robustness GL_EXT_texture_sRGB_decode GL_EXT_draw_buffers_indexed GL_EXT_texture_border_clamp GL_EXT_texture_buffer GL_EXT_copy_image GL_EXT_texture_cube_map_array GL_EXT_multi_draw_indirect GL_EXT_draw_elements_base_vertex GL_EXT_texture_rg GL_EXT_protected_textures GL_EXT_sRGB Disabled Extensions GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent Disabled WebGL Extensions
Window system binding vendor Vivante Corporation Window system binding version 1.5 Window system binding extensions EGL_KHR_fence_sync EGL_KHR_reusable_sync EGL_KHR_wait_sync EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_EXT_image_dma_buf_import EGL_EXT_image_dma_buf_import_modifiers EGL_KHR_lock_surface EGL_KHR_create_context EGL_KHR_surfaceless_context EGL_EXT_create_context_robustness EGL_EXT_protected_surface EGL_EXT_protected_content EGL_EXT_buffer_age EGL_ANDROID_native_fence_sync EGL_WL_bind_wayland_display EGL_WL_create_wayland_buffer_from_image EGL_KHR_partial_update EGL_EXT_swap_buffers_with_damage EGL_KHR_swap_buffers_with_damage Direct rendering Yes Reset notification strategy 0x8252 GPU process crash count 0 Compositor Information Tile Update Mode One-copy Partial Raster Enabled GpuMemoryBuffers Status ATC Software only ATCIA Software only DXT1 Software only DXT5 Software only ETC1 Software only R_8 Software only R_16 Software only RG_88 Software only BGR_565 Software only RGBA_4444 Software only RGBX_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE RGBA_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE BGRX_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE BGRX_1010102 Software only RGBX_1010102 Software only BGRA_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE RGBA_F16 Software only YVU_420 Software only YUV_420_BIPLANAR Software only UYVY_422 Software only Display(s) Information Info Display[0] bounds=[0,0 3840x2160], workarea=[0,0 3840x2160], scale=1, external. Color space information {primaries:BT709, transfer:IEC61966_2_1, matrix:RGB, range:FULL} Bits per color component 8 Bits per pixel 24 Video Acceleration _Information

richard-hu commented 3 years ago

Hello, @VaporMontey :

Thanks for your feedback.

Could i ask some questions to clarify this problem?

  1. Do you play 4k video in chromium browser?
  2. If the answer of the first question is "not", could you share what is the command you use to play 4k video.

The x264/HEVC HW video decoder in i.mx SOC is wrapped as a gstreamer plugin. The native way to utilize HW video decoder is to use gplay or gst-launch command. Please refer to: https://github.com/TechNexion/u-boot-tn-imx/wiki/Video-playback-with-HW-decoder-for-i.mx8

Another topic is if video HW decoder works in chromium? Duo to lack of a backend to glue gstreamer to chromium. Video HW decoder can't work in chromium.

There are some open-source projects that try to enable HW video decoder in chromium:

  1. chromium + NXP native VPU API: https://github.com/Freescale/chromium-imx
  2. chromium + PPAPI + gstreamer: https://github.com/kuscsik/gstreamer_pppapi https://github.com/Samsung/ChromiumGStreamerBackend

BR,

Richard

VaporMontey commented 3 years ago

Hi @richard-hu ,

Thanks for the response, I was trying to play a 4K stream in chromium browser and that's when the issue happened. Is HW decoding for chromium in program or it'll remain like this?

Also, if I have to use those solution to enable HW decode on Chromium I have to patch the source of chromium-ozone-wayland, where could I find it? (Hopefully those project work, giving a quick look they seem pretty old, advertised to work with Chromium <50)

There are some open-source projects that try to enable HW video decoder in chromium: chromium + NXP native VPU API: https://github.com/Freescale/chromium-imx chromium + PPAPI + gstreamer: https://github.com/kuscsik/gstreamer_pppapi https://github.com/Samsung/ChromiumGStreamerBackend

Best regards, Dario

richard-hu commented 3 years ago

@VaporMontey

Is HW decoding for chromium in program or it'll remain like this? No. HW decoding for chromium is not enabled by default in NXP i.mx BSP.

I have not tried to enable i.mx VPU in chromium in practice. Just share my understanding with you.

It looks the possible way is to integrate i.mx VPU library into chromium, like chromium-imx project. Although this project aims at i.mx6, but it should be similar for i.mx8MQ. Because both of they utilize i.mx VPU library to achieve VPU decoding.

For the existing Yocto 2.5(Sumo), chromium(chromium-ozone-wayland_71.0.3545) is proviced by meta-browser layer.

But you can find another chromium: chromium-imx in meta-browser/recipes-browser/chromium/chromium.inc which is the NXP-specific chromium. It refers to https://github.com/Freescale/chromium-imx
(The version of chromium is old.It's chromium 48) (Note: it’s ‘chromium-imx’, not ‘chromium’. ‘chromium-imx’ is the project based on ‘chromium’ but apply additional NXP-specific patchs to enable GPU rendering and VPU decode.)

In Yocto 2.5(sumo), chromium-imx is deprecated. So you have to add it back, and enable i.mx VPU for it.

  1. Add chromium-imx you can refer to the older Yocto (Yocto 2.4 Rocko) to enable it. https://github.com/Freescale/meta-freescale/tree/rocko/dynamic-layers/browser-layer/recipes-browser/chromium

  2. Enable i.mx VPU for chromium As what you can see in the link below, in Yocto 2.4(rocko), chromium-imx enables i.mx VPU by default. https://github.com/Freescale/meta-freescale/blob/rocko/dynamic-layers/browser-layer/recipes-browser/chromium/chromium-wayland_48.0.2548.0.bbappend

CHROMIUM_IMX_VPU_PATCHES_imxvpu += "${VPU_PATCHES}" (Note. It's for i.mx6, not for i.mx8. But it should be similar.)

But, in Yocto 2.5(sumo), i.mx VPU is left as an option and is not enabled by default. So you have to define the variable CHROMIUM_IMX_VPU_PATCHES to enable i.mx VPU. meta-freescale/dynamic-layers/browser-layer/recipes-browser/chromium/chromium-imx.inc

There are two key points to impact the performance of movie playback in chromium:

  1. Utilize i.mx VPU library in chromium.
  2. Zero-copy for the decoded frame.

It looks someone has enabled i.mx VPU decode (i.mx6) for chromium.
https://www.yoctoproject.org/pipermail/meta-freescale/2017-June/020602.html

BR,

Richard

VaporMontey commented 2 years ago

I'm closing this issue as I'm not using Yocto but Android.

Best regards