NV12/RGB4 rendering on Weston compositor

[snahulan]

[Ubuntu] Seen on multiple platforms Error seen: "Can't wrap flink to wl_buffer"

/opt/intel/mediasdk/share/mfx/samples/sample_decode h265 -hw -vaapi -i /home/tgl-deb/8k-Videos/Bunny_3840x2160_15mbps_60fps_Main_at_L5.1.265 -rwld -rgb4 libva info: VA-API version 1.12.0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_10 libva info: va_openDriver() returns 0 Decoding Sample Version 8.4.27.0

Input video HEVC Output format RGB4 (using vpp) Input: Resolution 3840x2160 Crop X,Y,W,H 0,0,3840,2160 Output: Resolution 3840x2160 Frame rate 60.00 Memory type vaapi MediaSDK impl hw MediaSDK version 1.35

Decoding started d_fps: 0.000, fwrite_fps: 0.000

Can't wrap flink to wl_buffer

DeliverOutput return error = -1 fread_fps: 0.000, fwrite_fps: 4830.918 Frame number: 1, fps: 0.791, fread_fps: 0.000, fwrite_fps: 4830.918 Decoding finished

[dmitryermilov]

Hi @SiewHoon , Can you please help? Does the issue look familiar?

[SiewHoon]

I can't replicate the issue on my system. Yes, I can help look on it.

Maybe need to know about version or commit id for libva, MSDK library, media-driver, wayland, weston using by reporter side.

[snahulan]

VA-API version: 1.12 (libva 2.10.0) --> libva and vaapi version Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.2.2 (0133e252) weston 9.0.0 intel-mediasdk-21.2.1 (commit ID: df004b9c)

Please let me know if you need any more info. Thanks

[SiewHoon]

@snahulan

Is this sample_decode rendering in XWayland? Or Wayland Weston with backend=drm-backend.so?

I can't reproduce this issue on my side with latest tag and same tag intel-mediasdk-21.2.1 or intel-mediasdk-21.2.2 version. Do you got build the your MSDK library with -DENABLE_WAYLAND=ON? Do you able to find see this libmfx_wayland.so?

I'm build mediasdk install directly to system path.

cmake .. -DCMAKE_INSTALL_PREFIX=/usr The sample app is install in this path : /usr/share/mfx/samples And libmfx_wayland.so also got install in /usr/share/mfx/samples as well.

To rendering in wayland side, I will do export LD_LIBRARY_PATH=/usr/share/mfx/samples because libmfx_wayland.so is install at this location.

Here is output from my TGL system: root@intel-corei7-64:/usr/share/mfx/samples# ./sample_decode h265 -i /home/weston/bunny.265 -hw -vaapi -rwld -rgb4 libva info: VA-API version 1.12.0 libva info: User environment variable requested driver 'iHD' libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_12 libva info: va_openDriver() returns 0 Decoding Sample Version 8.4.27.0

Input video HEVC Output format RGB4 (using vpp) Input: Resolution 3840x2160 Crop X,Y,W,H 0,0,3840,2160 Output: Resolution 3840x2160 Frame rate 60.00 Memory type vaapi MediaSDK impl hw MediaSDK version 1.35

Decoding started Frame number: 2938, fps: 59.888, fread_fps: 0.000, fwrite_fps: 60.111 Decoding finished root@intel-corei7-64:/usr/share/mfx/samples#

[snahulan]

Hi Siew Hoon,

We followed your steps of configuring and building with -DENABLE_WAYLAND=ON. We are able to see libmfx_wayland.so, and we exported LD_LIBRARY_PATH as well. ** Media SDK Configuration Summary ** Enable: ENABLE_OPENCL : ON ENABLE_X11 : ON ENABLE_X11_DRI3 : ON ENABLE_WAYLAND : ON

Still unable to decode on wayland. Still seeing this issue: Can't wrap flink to wl_buffer

[SiewHoon]

Hi @snahulan .

Do you able to give a try on using gst-msdk to check see can rendering or not?

gst-launch-1.0 filesrc location=/home/tgl-deb/8k-Videos/Bunny_3840x2160_15mbps_60fps_Main_at_L5.1.265 ! h265parse ! msdkh265dec ! glimagesink

or

gst-launch-1.0 filesrc location=/home/tgl-deb/8k-Videos/Bunny_3840x2160_15mbps_60fps_Main_at_L5.1.265 ! h265parse ! msdkh265dec ! waylandsink

Is that possible save the weston loading log as well?
What is kernel version that you are using and libdrm version?

[snahulan]

Hi Siew Hoon,

In wayland mode: Tried with the first cmd, observed some corruption. Tried the 2nd cmd, it could render normally with some lag. So gstreamer is able to work, but msdk is still giving that error. This could mean that it's related to msdk (app layer)?

libdrm version: (tag: libdrm-2.4.106) Bump version to 2.4.106

Kernel version: Linux tgldeb-Tiger-Lake-Client-Platform 5.13.0-rc3-cidrm10128+ #1 SMP PREEMPT Tue May 25 14:19:34 +08 2021 x86_64 x86_64 x86_64 GNU/Linux

Attached the weston loading log: weston_log.txt

[SiewHoon]

From weston_log.txt, I can see the issue at here. Your weston backend didnt success loading using drm-backend.so. It is fall back to gl-renderer.so. This drm-backend.so is fail to loading is due to drm-backend is didn't get destroy, It is using by your gnome X.

I didnt run sample application with weston backend=gl-renderer.so at here. We always run using drm-backend.so in native wayland weston or fallback using Xwayland to run it.

Step 1: Boot up your Ubuntu system into console mode. Run this command "systemctl set-default multi-user.target" and reboot the system.

Step 2: set XDG_RUNTIME_DIR Edit .bashrc file in ~/ put in this few lines to set for XDG_RUNTIME_DIR if you don't know where to set path use for XDG_RUNTIME_DRI.

if test -z "${XDG_RUNTIME_DIR}"; then export XDG_RUNTIME_DIR=/tmp/${UID}-runtime-dir if ! test -d "${XDG_RUNTIME_DIR}"; then mkdir "${XDG_RUNTIME_DIR}" chmod 0700 "${XDG_RUNTIME_DIR}" fi fi

Step 3: copy weston.ini put into home directiory side: ~/.config/ weston.ini file you can find in weston source code based side. Just remember to set "backend=drm-backend.so", so that it is using drm-backend.so.

Step 4: Copy this weston.service file put into /lib/systemd.system cd /lib/systemd/system, copy and paste this weston.service into system folder. And modify User who is using that on that system and working directory.

(This weston.service file, copy from https://coral.googlesource.com/weston-imx-debian/+/refs/heads/release-beaker/debian/lib/systemd/system/weston.service and modify it to use tty1 instead of tty7 ).

Step 5: sudo systemctl enable weston.service sudo systemctl status weston.service

Step 6: Fall back to gnome desktop To disable weston, you can run this "sudo systemctl disable weston.service" command: systemctl set-default graphical.target and reboot system and login.

[snahulan]

Hi Siew Hoon,

I'm still attempting to rebuild Intel Weston-wayland compositor. But we tried on another OS (yocto) and we compared the Weston boot log.

This is the difference when it is loading the drm-backend.so: weston_log_yocto.txt

Does this still look like drm-backend is failing to load?

[snahulan]

Hi all,

After rebuilding the media stack as per Siew Hoon's steps (as user), I'm able to run the rgb4 rendering on wayland and I dont see the error anymore. Also needed to export LD_LIBRARY_PATH to :~/mediasdk/lib:~/mediasdk/plugins:~/mediasdk/samples:~/mediasdk/share/mfx/samples

This ticket can be closed. Thank you

[dmitryermilov]

Good to know. Special thanks to Siew Hoon!