search

termux / termux-packages

A package build system for Termux.
https://termux.dev
Other
13.06k stars 3k forks source link

[Package]: Enable vulkan drivers (swrast, freedreno) in mesa #15271

Closed chaitan3 closed 1 year ago

chaitan3 commented 1 year ago

Why do you want to add this package to Termux?

Following up from the issue on adding a generic vulkan loader to termux: https://github.com/termux/termux-packages/issues/15266

Mesa has a few vulkan drivers that can be used to provide alternative vulkan implementations separate from the ones provided by the android system. Two drivers seem to be good candidates:

  1. lavapipe (swrast)
  2. turnip with kgsl (freedreno)

Some more info can be found on the following links: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4738 https://archlinuxarm.org/packages/aarch64/mesa/files/PKGBUILD

Flags for building turnip and swrast

-Dgallium-drivers=swrast,zink \
    -Dvulkan-drivers=swrast,freedreno \
    -Dfreedreno-kgsl=true \
    -Dzlib=enabled

Home page URL

No response

Source code URL

https://gitlab.freedesktop.org/mesa/mesa

Packaging policy acknowledgement

Additional information

No response

truboxl commented 1 year ago

Need root to work. May even need manual installation from user to adjust SONAME on their own. We may even brick their devices if not careful.

chaitan3 commented 1 year ago

Ah, did not realize it required root. I saw some posts online and discussions on discord that mentioned it could be possible:

  1. https://www.reddit.com/r/termux/comments/p7i5kg/world_of_warcraft_wrath_of_the_lich_king_with/
  2. https://github.com/bylaws/libadrenotools
  3. https://github.com/skyline-emu/skyline/actions/runs/1826247099

The kgsl handle seems to have user permissions.

crw-rw-rw- 1 system system 502, 0 Feb 20 19:39 /dev/kgsl-3d0

If it can't be done in plain termux, then this issue can be closed. Also, even if turnip does not work, it might be worth enabling lavapipe for phones that do not have vulkan?

licy183 commented 1 year ago

Yeah, I can confirm that /dev/kgsl-3d0 can be accessed on my device (both Android 9 and Android 12).

licy183 commented 1 year ago

OK. I can confirm that lavapipe over zink can be used for glmark2.

But there is another question. If we enable kgsl for freedreno, then we must disable egl for mesa (which needs dri2 and dri2 is conflict with kgsl currently) and that means we have to switch glx back to xlib and glvnd cannot be used too.

truboxl commented 1 year ago

I may have misinterpreted the root requirement. As I understand from https://docs.mesa3d.org/android.html#replacing-android-drivers-on-stock-android, you need to replace drivers in /vendor which requires root access. But I think that's for replacing the entire Android system to use the mesa drivers.

Users might need to patch out DT_RUNPATH and correct SONAME if they go for manually replacing Android system drivers with mesa drivers route though. Device bricking is imminent for this. This is probably outside the scope of Termux too.

Other than that I believe no root required AFAIK.

Since /dev/kgsl-3d0 is accessible, that may explain why rootless proot environment works since the glibc drivers talk using ioctl() so no bionic libc restriction. You also can try cat /dev/kgsl-3d0 to get OH HAI GPU in both Termux and proot Linux environments.

This might be workable.

licy183 commented 1 year ago

Yeah, it (freedreno with kgsl) works with vulkaninfo. But it cannot compile in the same build.sh with mesa because enable gallium-drivers will mark system_has_kms_drm as false.

freedreno with kgsl cannot work over zink because zink requires KHR_timeline_semaphore but kgsl doesn't support it.

I've compiled Vulkan-ExtensionLayer for freedreno with kgsl, but it doesn't work for me. There is always blank screen. More investigations are needed.

FYI: https://github.com/termux/termux-packages/actions/runs/4230928181.

Heasterian commented 1 year ago

Question is if after this merge building with kgsl will still be issue: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394

About rendering Vulkan in Termux, there is a chance that you need MESA_VK_WSI_DEBUG=sw env variable to get anything except black screen as afaik, we don't have DRI3 support. Zink with Freedreno can run without it if you use version from before Kopper merge, but not Vulkan itself.

chaitan3 commented 1 year ago

The build instructions over here have egl enabled for mesa turnip kgsl, maybe some patches from that repo can be borrowed https://github.com/Grima04/mesa-turnip-kgsl

If not, the mesa PR from the previous comment could help

licy183 commented 1 year ago

Yeah, I can get freedreno with kgsl work on Termux, but it is not able to work for zink anyway. After kopper gets merged, WSI is needed to make zink work properly but freedreno with kgsl won't work for WSI.

IntinteDAO commented 1 year ago

Yeah, I can get freedreno with kgsl work on Termux, but it is not able to work for zink anyway. After kopper gets merged, WSI is needed to make zink work properly but freedreno with kgsl won't work for WSI.

Maybe we should write to Mesa Repository about our need? I write to Mike that create Kopper and he's propose that.

Heasterian commented 1 year ago

@IntinteDAO I think that recent xserver versions can run zink with Kopper using MESA_VK_WSI_DEBUG=sw env variable, but I'm not sure about older ones.

Heasterian commented 1 year ago

Afaik it's supposed to be solution for cases like ours, where we don't have DRI3.

IntinteDAO commented 1 year ago

@IntinteDAO I think that recent xserver versions can run zink with Kopper using MESA_VK_WSI_DEBUG=sw env variable, but I'm not sure about older ones.

Forgive me for asking, but what difference does it make to the average user? In the sense we want Zink in Termux, if your way works then we need to make sure it works for us. And if someone wants to use proot / chroot then they can use virglrenderer and that should work.

ThatMG393 commented 1 year ago

https://github.com/ThatMG393/gpu_accel_termux

this basically does it for you.

licy183 commented 1 year ago

Yeah, I can get freedreno with kgsl work on Termux, but it is not able to work for zink anyway. After kopper gets merged, WSI is needed to make zink work properly but freedreno with kgsl won't work for WSI.

Maybe we should write to Mesa Repository about our need? I write to Mike that create Kopper and he's propose that.

I agree with this.

licy183 commented 1 year ago

https://github.com/ThatMG393/gpu_accel_termux

this basically does it for you.

Emmm... This is actually nothing to do with the kgsl vulkan driver... This link provides a way to run zink with the latest mesa.

ThatMG393 commented 1 year ago

https://github.com/ThatMG393/gpu_accel_termux this basically does it for you.

Emmm... This is actually nothing to do with the kgsl vulkan driver... This link provides a way to run zink with the latest mesa.

You can enable KGSL in the while building

IntinteDAO commented 1 year ago

I agree with this.

Do you want to do this? I can, but I'm not a good dev and you can write more info about our problem

licy183 commented 1 year ago

I agree with this.

Do you want to do this? I can, but I'm not a good dev and you can write more info about our problem

Emmm... I've submitted it upstream. https://gitlab.freedesktop.org/mesa/mesa/-/issues/8460

IntinteDAO commented 1 year ago

I agree with this.

Do you want to do this? I can, but I'm not a good dev and you can write more info about our problem

Emmm... I've submitted it upstream. #15271 (comment).

Forgive me, because now I don't understand myself :P

The idea is to write on the Mesa gitlab about the fact that there is a project like Termux and because of Kopper we cannot use Zink for the new Mesa. Maybe someone can help us from there (at least in theory).

Unless there is such a thread.

licy183 commented 1 year ago

I agree with this.

Do you want to do this? I can, but I'm not a good dev and you can write more info about our problem

Emmm... I've submitted it upstream. #15271 (comment).

Forgive me, because now I don't understand myself :P

The idea is to write on the Mesa gitlab about the fact that there is a project like Termux and because of Kopper we cannot use Zink for the new Mesa. Maybe someone can help us from there (at least in theory).

Unless there is such a thread.

Emmm... I put the wrong link... https://gitlab.freedesktop.org/mesa/mesa/-/issues/8460

IntinteDAO commented 1 year ago

@licy183 Thanks

Heasterian commented 1 year ago

With new Termux-X11 using XCB and git version of Mesa Zink and Turnip with KGSL runs fine, but you need to compile both KGSL and MSM (compilation of just KGSL fails, I'll report that on Mesa git tomorrow). I tested it in proot, but it should be the same for Termux.

name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Mesa (0x5143)
    Device: zink Vulkan 1.3(Turnip Adreno (TM) 650 (MESA_TURNIP)) (0x6050002)
    Version: 23.1.0
    Accelerated: yes
    Video memory: 8618MB
    Unified memory: yes
    Preferred profile: core (0x1)
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
Memory info (GL_ATI_meminfo):
    VBO free memory - total: 8612 MB, largest block: 8612 MB
    VBO free aux. memory - total: 0 MB, largest block: 0 MB
    Texture free memory - total: 8612 MB, largest block: 8612 MB
    Texture free aux. memory - total: 0 MB, largest block: 0 MB
    Renderbuffer free memory - total: 8612 MB, largest block: 8612 MB
    Renderbuffer free aux. memory - total: 0 MB, largest block: 0 MB
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 8618 MB
    Total available memory: 8618 MB
    Currently available dedicated video memory: 8612 MB
OpenGL vendor string: Mesa
OpenGL renderer string: zink Vulkan 1.3(Turnip Adreno (TM) 650 (MESA_TURNIP))
OpenGL core profile version string: 4.6 (Core Profile) Mesa 23.1.0-devel (git-f99eab23ad)
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6 (Compatibility Profile) Mesa 23.1.0-devel (git-f99eab23ad)
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 23.1.0-devel (git-f99eab23ad)
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
Heasterian commented 1 year ago

And you need to use MESA_VK_WSI_DEBUG=sw I talked about earlier to get it to run.

Heasterian commented 1 year ago

Looks like compilation with only kgsl is untested with X11 and Zink and driver compiled with msm and kgsl together also don't work correctly so while I was able even get pure vulkan to work with software WSI, it's crashing (vulkaninfo don't even work, DX9 demo found in web is crashing after instant, vkcube works fine, zink app also are freezing after moment). I'll report it compilation errors on Mesa gitlab and when compiling just KGSL will be possible, I'll retest.

licy183 commented 1 year ago

vulkaninfo doesn't work for vnc with x11 platform. IIUC this is not a bug of Mesa on Termux, because the same question exists on Ubuntu 22.04.

licy183 commented 1 year ago

Anyway, I've enabled the kgsl vulkan driver. It should work with software WSI (just set envvar MESA_VK_WSI_DEBUG=sw as Heasterian says).

twaik commented 1 year ago

Actually I've patched mesa to make it work without MESA_VK_WSI_DEBUG=sw. mesa.patch.txt. It works in proot/chroot so I do not see any problem with using it in termux. But it is not possible to build both turnip and mesa simultaneously since build script disables drm backend when selecting freedreno vulkan drivers. We were building mesa+turnip as follows

meson buildturnip -D platforms=x11 -D glx=disabled -D egl=disabled -D gallium-drivers= -D vulkan-drivers=freedreno -D freedreno-kmds=kgsl -D dri3=enabled

meson buildzink -D platforms=x11 -D gallium-drivers=zink,swrast -D vulkan-drivers= -D dri3=enabled -D glx=dri -D egl=enabled -D gles1=enabled -D gles2=enabled
twaik commented 1 year ago

*I patched mesa to make zink work without MESA_VK_WSI_DEBUG=sw.

twaik commented 1 year ago

Can we enable imagination-experimental driver? I've enabled it locally, but it is not loaded with the following error.

~ $ vkcube
vkEnumerateInstanceExtensionProperties failed to find the VK_KHR_xcb_surface extension.

Do you have a compatible Vulkan installable client driver (ICD) installed?
Please look at the Getting Started guide for additional information.

My PowerVR device has /dev/dri/card0 and /dev/dri/renderD128 so I think it should/can be supported.

licy183 commented 1 year ago

IIRC /dev/dri/* is not able to access without root permission...

twaik commented 1 year ago

Both card0 and renderD128 are world writable so they can be accessed.

~ $ ls -la /dev/dri
total 0
drwxr-xr-x  2 root root           80 Jun 25 16:50 .
drwxr-xr-x 27 root root         4220 Jun 26 19:43 ..
crw-rw-rw-  1 root graphics 226,   0 Jun 25 16:50 card0
crw-rw-rw-  1 root graphics 226, 128 Jun 25 16:50 renderD128
licy183 commented 1 year ago

Oops, but they are not accessable on my device. I think we can enable it for testing, but I don't have a device with PowerVR GPU...

twaik commented 1 year ago

You can use mine.

twaik commented 1 year ago

But again, it is not loaded because of lack of VK_KHR_xcb_surface extension. But it is pretty weird, it does not implement special wsi, it uses mesa's vulkan common wsi mechanisms.

licy183 commented 1 year ago

Emmm... I think we should wait until this driver is stable before enabling it...

twaik commented 1 year ago

I just wanted to make some dri3 stuff for turnip to make it zero-copy (or at least single-copy, copying while it is being composed can't be avoided).

twaik commented 1 year ago

I've patched imagination driver to report VK_KHR_xcb_surface support, but still has no luck. src-imagination-vulkan-pvr_device.c.patch.txt

~ $ vkcube
vkEnumeratePhysicalDevices reported zero accessible devices.

Do you have a compatible Vulkan installable client driver (ICD) installed?
Please look at the Getting Started guide for additional information.
IntinteDAO commented 1 year ago

I have a question about this issue.

Well, I don't know if it's a system issue or something else, but no Vulkan option works for me except vulkan Android.

I don't know how to force swrast or freedreno (I just kept it and removed the rest) or potentially swiftshader, but I keep getting the same error.

Obviously it's worth finding out if it's a smartphone problem or if it can be 'fixed'.

WARNING: [Loader Message] Code 0 : loader_scanned_icd_add: Using deprecated ICD interface of 'vkGetInstanceProcAddr' instead of 'vk_icdGetInstanceProcAddr' for ICD /data/data/com.termux/files/usr/lib/libvulkan.so
WARNING: [Loader Message] Code 0 : loader_scanned_icd_add: Driver /data/data/com.termux/files/usr/lib/libvulkan.so supports Vulkan 1.3, but only supports loader interface version 0. Interface version 5 or newer required to support this version of Vulkan (Policy #LDP_DRIVER_7)
ERROR: [Loader Message] Code 0 : loader_add_instance_extensions: /data/data/com.termux/files/usr/lib/libvulkan.so's vkEnumerateInstanceExtensionProperties points to the loader, this would lead to infinite recursion.
ERROR: [Loader Message] Code 0 : loader_add_instance_extensions: /data/data/com.termux/files/usr/lib/libvulkan.so's vkEnumerateInstanceExtensionProperties points to the loader, this would lead to infinite recursion.

Vulkaninfo. vkcube doesn't work, but without any logs

licy183 commented 1 year ago

I have a question about this issue.

Well, I don't know if it's a system issue or something else, but no Vulkan option works for me except vulkan Android.

I don't know how to force swrast or freedreno (I just kept it and removed the rest) or potentially swiftshader, but I keep getting the same error.

Obviously it's worth finding out if it's a smartphone problem or if it can be 'fixed'.

WARNING: [Loader Message] Code 0 : loader_scanned_icd_add: Using deprecated ICD interface of 'vkGetInstanceProcAddr' instead of 'vk_icdGetInstanceProcAddr' for ICD /data/data/com.termux/files/usr/lib/libvulkan.so
WARNING: [Loader Message] Code 0 : loader_scanned_icd_add: Driver /data/data/com.termux/files/usr/lib/libvulkan.so supports Vulkan 1.3, but only supports loader interface version 0. Interface version 5 or newer required to support this version of Vulkan (Policy #LDP_DRIVER_7)
ERROR: [Loader Message] Code 0 : loader_add_instance_extensions: /data/data/com.termux/files/usr/lib/libvulkan.so's vkEnumerateInstanceExtensionProperties points to the loader, this would lead to infinite recursion.
ERROR: [Loader Message] Code 0 : loader_add_instance_extensions: /data/data/com.termux/files/usr/lib/libvulkan.so's vkEnumerateInstanceExtensionProperties points to the loader, this would lead to infinite recursion.

Vulkaninfo. vkcube doesn't work, but without any logs

You can open a new issue for it and provide the termux-info output.

licy183 commented 1 year ago

This issue should have been fixed. lavapipe and freedreno has been added.

romanovj commented 1 year ago

@licy183 is it possible to have vulkan video decoder? VK_KHR_video_decode_queue extension

licy183 commented 1 year ago

Not possible for now. Neither lavapipe nor freedreno ksgl implements this extension.

truboxl commented 1 year ago

Not even Android support Vulkan Video extension https://source.android.com/docs/compatibility/13/android-13-cdd#7142_vulkan

[C-1-11] MUST NOT enumerate support for the VK_KHR_video_queue, VK_KHR_video_decode_queue, or VK_KHR_video_encode_queue extensions.

youmukonpaku1337 commented 6 months ago

oh wow these patches allow compiling latest mesa :D, thanks xmem