search

bylaws / libadrenotools

A library for applying rootless Adreno GPU driver modifications/replacements
BSD 2-Clause "Simplified" License
301 stars 45 forks source link

Asking for help getting it to work on a test app #19

Closed darksylinc closed 7 months ago

darksylinc commented 7 months ago

I'm playing with AdrenoTools but I can't get it to work on a Xiaomi POCO F2 Pro (Adreno 640) running stock ROM Android 12.

I know that it is supposed to work fine on my phone because PPSSPP manages to load the driver fine, reports a different driver string when using the alterante driver and even displays graphical glitches when using this other driver.

I can't figure out what kotlin settings, linker flags or whatever else I'm doing wrong compared to PPSSPP.

I prepared an extremely simple repro test here.

It's a very simple sample that uses GameActivity as backbone (perhaps that's the problem??? I'm also having the same issue using NativeActivity).

No matter what I do I get:

NO VK DEVICES!

adrenotools_open_libvulkan returns a valid ptr but then vkEnumeratePhysicalDevices returns VK_SUCCESS with numDevices = 0.

The full log:

---------------------------- PROCESS STARTED (24589) for package com.example.adrenotoolstest2 ----------------------------
2024-03-22 20:49:56.652 24589-24589 Compatibil...geReporter com.example.adrenotoolstest2         D  Compat change id reported: 171979766; UID 10546; state: ENABLED
2024-03-22 20:49:56.670 24589-24589 nativeloader            com.example.adrenotoolstest2         D  Configuring clns-4 for other apk /data/app/~~BHNx5TR2rgoz9pbGpidccg==/com.example.adrenotoolstest2-N_zkCmM2qiQ3wAUzEz-uPQ==/base.apk. target_sdk_version=34, uses_libraries=, library_path=/data/app/~~BHNx5TR2rgoz9pbGpidccg==/com.example.adrenotoolstest2-N_zkCmM2qiQ3wAUzEz-uPQ==/lib/arm64:/data/app/~~BHNx5TR2rgoz9pbGpidccg==/com.example.adrenotoolstest2-N_zkCmM2qiQ3wAUzEz-uPQ==/base.apk!/lib/arm64-v8a, permitted_path=/data:/mnt/expand:/data/user/0/com.example.adrenotoolstest2
2024-03-22 20:49:56.680 24589-24589 GraphicsEnvironment     com.example.adrenotoolstest2         V  ANGLE Developer option for 'com.example.adrenotoolstest2' set to: 'default'
2024-03-22 20:49:56.680 24589-24589 GraphicsEnvironment     com.example.adrenotoolstest2         V  Updatable production driver is not supported on the device.
2024-03-22 20:49:56.682 24589-24589 ForceDarkHelperStubImpl com.example.adrenotoolstest2         I  initialize for com.example.adrenotoolstest2 , ForceDarkOrigin
2024-03-22 20:49:56.683 24589-24589 OpenGLRenderer          com.example.adrenotoolstest2         D  JNI_OnLoad success
2024-03-22 20:49:56.684 24589-24589 MiuiForceDarkConfig     com.example.adrenotoolstest2         I  setConfig density:2.750000, mainRule:0, secondaryRule:0, tertiaryRule:0
2024-03-22 20:49:56.687 24589-24589 NetworkSecurityConfig   com.example.adrenotoolstest2         D  No Network Security Config specified, using platform default
2024-03-22 20:49:56.688 24589-24589 NetworkSecurityConfig   com.example.adrenotoolstest2         D  No Network Security Config specified, using platform default
2024-03-22 20:49:56.698  3128-3576  ActivityManagerWrapper  com.mi.android.globallauncher        E  getRecentTasks: taskId=1855   userId=0   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.adrenotoolstest2/com.example.adrenotoolstest2.MainActivity} }
2024-03-22 20:49:56.704 24589-24589 AppCompatDelegate       com.example.adrenotoolstest2         D  Checking for metadata for AppLocalesMetadataHolderService : Service not found
2024-03-22 20:49:56.710 24589-24589 ViewScrollStubImpl      com.example.adrenotoolstest2         D  the value of ScrollFrictionRes is: 0.015
2024-03-22 20:49:56.715 24589-24589 IS_CTS_MODE             com.example.adrenotoolstest2         D  false
2024-03-22 20:49:56.715 24589-24589 MULTI_WIND...CH_ENABLED com.example.adrenotoolstest2         D  false
2024-03-22 20:49:56.716 24589-24589 IS_CTS_MODE             com.example.adrenotoolstest2         D  false
2024-03-22 20:49:56.716 24589-24589 MULTI_WIND...CH_ENABLED com.example.adrenotoolstest2         D  false
2024-03-22 20:49:56.718 24589-24589 DecorView[]             com.example.adrenotoolstest2         D  getWindowModeFromSystem  windowmode is 1
2024-03-22 20:49:56.732 24589-24589 drenotoolstest          com.example.adrenotoolstest2         W  Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (unsupported, reflection, allowed)
2024-03-22 20:49:56.732 24589-24589 drenotoolstest          com.example.adrenotoolstest2         W  Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (unsupported, reflection, allowed)
2024-03-22 20:49:56.739 24589-24589 GameActivity            com.example.adrenotoolstest2         I  Looking for library libadrenotoolstest2.so
2024-03-22 20:49:56.739 24589-24589 GameActivity            com.example.adrenotoolstest2         I  Found library libadrenotoolstest2.so. Loading...
2024-03-22 20:49:56.741 24589-24589 GameActivity            com.example.adrenotoolstest2         D  GameActivity_register
2024-03-22 20:49:56.741 24589-24589 GameActivity            com.example.adrenotoolstest2         V  Registering com/google/androidgamesdk/GameActivity's 21 native methods...
2024-03-22 20:49:56.741 24589-24589 threaded_app            com.example.adrenotoolstest2         V  Creating: 0xb40000709caaf300
2024-03-22 20:49:56.741 24589-24589 threaded_app            com.example.adrenotoolstest2         V  Callbacks set: 0xb40000709caaf350
2024-03-22 20:49:56.741 24589-24589 threaded_app            com.example.adrenotoolstest2         V  Launching android_app_entry in a thread
2024-03-22 20:49:56.741 24589-24616 threaded_app            com.example.adrenotoolstest2         V  android_app_entry called
2024-03-22 20:49:56.741 24589-24616 threaded_app            com.example.adrenotoolstest2         V  android_app = 0xb40000709cb1f600
2024-03-22 20:49:56.741 24589-24616 threaded_app            com.example.adrenotoolstest2         V  config = 0xb4000070a6409080
2024-03-22 20:49:56.741 24589-24616 threaded_app            com.example.adrenotoolstest2         V  activity = 0xb40000709caaf300
2024-03-22 20:49:56.741 24589-24616 threaded_app            com.example.adrenotoolstest2         V  assetmanager = 0xb40000709caadf00
2024-03-22 20:49:56.741 24589-24616 threaded_app            com.example.adrenotoolstest2         V  Config: mcc=0 mnc=0 lang=es cnt=US orien=1 touch=3 dens=440 keys=1 nav=1 keysHid=3 navHid=0 sdk=31 size=2 long=2 modetype=1 modenight=1
2024-03-22 20:49:56.744 24589-24616 DriverReplacer          com.example.adrenotoolstest2         I  DRIVER REPLACEMENT LOADED
2024-03-22 20:49:56.744 24589-24616 hook_impl               com.example.adrenotoolstest2         I  hook_android_load_sphal_library: filename: vulkan.adreno.so
2024-03-22 20:49:56.744 24589-24616 hook_impl               com.example.adrenotoolstest2         I  hook_android_dlopen_ext: filename: vulkan.adreno.so
2024-03-22 20:49:56.745 24589-24616 hook_impl               com.example.adrenotoolstest2         I  hook_android_load_sphal_library: filename: vulkan.kona.so
2024-03-22 20:49:56.745 24589-24616 hook_impl               com.example.adrenotoolstest2         I  hook_android_dlopen_ext: filename: vulkan.kona.so
2024-03-22 20:49:56.746 24589-24616 vulkan                  com.example.adrenotoolstest2         D  searching for layers in '/data/app/~~BHNx5TR2rgoz9pbGpidccg==/com.example.adrenotoolstest2-N_zkCmM2qiQ3wAUzEz-uPQ==/lib/arm64'
2024-03-22 20:49:56.747 24589-24616 vulkan                  com.example.adrenotoolstest2         D  searching for layers in '/data/app/~~BHNx5TR2rgoz9pbGpidccg==/com.example.adrenotoolstest2-N_zkCmM2qiQ3wAUzEz-uPQ==/base.apk!/lib/arm64-v8a'
2024-03-22 20:49:56.747 24589-24616 DriverReplacer          com.example.adrenotoolstest2         E  NO VK DEVICES!
2024-03-22 20:49:56.752 24589-24589 GameActivity            com.example.adrenotoolstest2         V  onStart_native
2024-03-22 20:49:56.752 24589-24589 threaded_app            com.example.adrenotoolstest2         V  Start: 0xb40000709caaf300
2024-03-22 20:49:56.752 24589-24616 threaded_app            com.example.adrenotoolstest2         V  activityState=10
2024-03-22 20:49:56.754 24589-24589 threaded_app            com.example.adrenotoolstest2         V  Resume: 0xb40000709caaf300
2024-03-22 20:49:56.754 24589-24616 threaded_app            com.example.adrenotoolstest2         V  activityState=11
2024-03-22 20:49:56.774 24589-24589 threaded_app            com.example.adrenotoolstest2         V  WindowInsetsChanged: 0xb40000709caaf300
2024-03-22 20:49:56.782 24589-24613 AdrenoGLES-0            com.example.adrenotoolstest2         I  QUALCOMM build                   : db3d445dbc, Ia06b22fa1a
                                                                                                    Build Date                       : 10/04/21
                                                                                                    OpenGL ES Shader Compiler Version: EV031.32.02.16
                                                                                                    Local Branch                     : 
                                                                                                    Remote Branch                    : 
                                                                                                    Remote Branch                    : 
                                                                                                    Reconstruct Branch               : 
2024-03-22 20:49:56.782 24589-24613 AdrenoGLES-0            com.example.adrenotoolstest2         I  Build Config                     : S P 10.0.7 AArch64
2024-03-22 20:49:56.782 24589-24613 AdrenoGLES-0            com.example.adrenotoolstest2         I  Driver Path                      : /vendor/lib64/egl/libGLESv2_adreno.so
2024-03-22 20:49:56.785 24589-24613 AdrenoGLES-0            com.example.adrenotoolstest2         I  PFP: 0x016dd093, ME: 0x00000000
2024-03-22 20:49:56.787 24589-24589 SurfaceView             com.example.adrenotoolstest2         D  UPDATE null, mIsCastMode = false
2024-03-22 20:49:56.789 24589-24589 GameActivity            com.example.adrenotoolstest2         V  onSurfaceCreated_native
2024-03-22 20:49:56.789 24589-24589 threaded_app            com.example.adrenotoolstest2         V  NativeWindowCreated: 0xb40000709caaf300 -- 0xb40000704006c010
2024-03-22 20:49:56.789 24589-24589 threaded_app            com.example.adrenotoolstest2         V  android_app_set_window called
2024-03-22 20:49:56.789 24589-24616 threaded_app            com.example.adrenotoolstest2         V  APP_CMD_INIT_WINDOW
2024-03-22 20:49:56.789 24589-24589 threaded_app            com.example.adrenotoolstest2         V  NativeWindowResized: 0xb40000709caaf300 -- 0xb40000704006c010 ( 1080 x 2246 )
2024-03-22 20:49:56.789 24589-24589 threaded_app            com.example.adrenotoolstest2         V  NativeWindowRedrawNeeded: 0xb40000709caaf300 -- 0xb40000704006c010
2024-03-22 20:49:56.795 24589-24613 hw-ProcessState         com.example.adrenotoolstest2         D  Binder ioctl to enable oneway spam detection failed: Invalid argument
2024-03-22 20:49:56.807 24589-24613 LB                      com.example.adrenotoolstest2         E  fail to open file: No such file or directory
2024-03-22 20:49:56.818 24589-24589 SurfaceView             com.example.adrenotoolstest2         D  UPDATE Surface(name=SurfaceView[com.example.adrenotoolstest2/com.example.adrenotoolstest2.MainActivity])/@0x618d2d4, mIsProjectionMode = false
2024-03-22 20:49:56.818 24589-24589 SurfaceControl          com.example.adrenotoolstest2         D  nativeSetScreenProjection ===> setScreenProjection
2024-03-22 20:49:56.818 24589-24589 SurfaceControl          com.example.adrenotoolstest2         D  nativeSetScreenProjection ===> setScreenProjection
2024-03-22 20:49:56.819 24589-24589 SurfaceControl          com.example.adrenotoolstest2         D  nativeSetScreenProjection ===> setScreenProjection
2024-03-22 20:49:56.823 24589-24589 threaded_app            com.example.adrenotoolstest2         V  WindowFocusChanged: 0xb40000709caaf300 -- 1
2024-03-22 20:49:56.824 24589-24589 DecorView[]             com.example.adrenotoolstest2         D  onWindowFocusChanged hasWindowFocus true
2024-03-22 20:49:56.836 24589-24589 DecorView[]             com.example.adrenotoolstest2         D  getWindowModeFromSystem  windowmode is 1
2024-03-22 20:49:56.836 24589-24589 DecorView[]             com.example.adrenotoolstest2         D  updateDecorCaptionStatus displayWindowDecor is false
2024-03-22 20:49:56.847 24589-24589 threaded_app            com.example.adrenotoolstest2         V  WindowInsetsChanged: 0xb40000709caaf300
2024-03-22 20:49:56.848 24589-24589 threaded_app            com.example.adrenotoolstest2         V  NativeWindowResized: 0xb40000709caaf300 -- 0xb40000704006c010 ( 1080 x 2400 )
2024-03-22 20:49:56.848 24589-24589 threaded_app            com.example.adrenotoolstest2         V  NativeWindowRedrawNeeded: 0xb40000709caaf300 -- 0xb40000704006c010

Debugging further, the first time the hook activates (when I call vkCreateInstance) it ends up inside hook_android_dlopen_ext:

[libhook_impl.so] hook_android_dlopen_ext(const char *, int, const android_dlextinfo *) hook_impl.cpp:39
[libhook_impl.so] hook_android_load_sphal_library(const char *, int) hook_impl.cpp:153
[libmain_hook.so] android_load_sphal_library main_hook.c:8
<unknown> 0x00000070b85a302c
<unknown> 0x00000070b85a03e4
<unknown> 0x00000070b8597cf0
<unknown> 0x00000070b8597068
[libadrenotoolstest2.so] replaceDriver(const char *) main.cpp:58
[libadrenotoolstest2.so] android_main(android_app *) main.cpp:115
[libadrenotoolstest2.so] android_app_entry(void *) android_native_app_glue.c:235
[libc.so] __pthread_start(void *) 0x0000007146114b18
[libc.so] __start_thread 0x00000071460b1360

It reaches the following:

    auto hookImpl{linkernsbypass_namespace_dlopen("libhook_impl.so", RTLD_NOW, driverNs)};
    if (!hookImpl)
        return nullptr;

But hookImpl is nullptr.

After that it gets called again (exact same callstack) and also ends up with hookImpl = nullptr, after that it gives up.

When this happens I noticed logcat prints:

hook_android_load_sphal_library: filename: vulkan.adreno.so
hook_android_dlopen_ext: filename: vulkan.adreno.so
hook_android_load_sphal_library: filename: vulkan.kona.so
hook_android_dlopen_ext: filename: vulkan.kona.so

This is a stark difference with PPSSPP which is successful and prints:

hook_android_load_sphal_library: filename: vulkan.adreno.so
hook_android_dlopen_ext: filename: vulkan.adreno.so
hook_android_dlopen_ext: applied libfile_redirect_hook
hook_android_dlopen_ext: loading custom driver: /data/user/0/org.ppsspp.ppsspp/files/drivers/Mesa Turnip Adreno Driver 22.3.1-r2/libvulkan_freedreno.so

I suspect perhaps the Vulkan driver gets loaded too soon; since I AFAIK the injection must happen before loading Vulkan.

But I tried modifying game-activity/native_app_glue/android_native_app_glue.c (which comes with the SDK) so that the injection is the first thing I do in Thread-1:

JNIEXPORT
void GameActivity_onCreate(GameActivity* activity, void* savedState,
                           size_t savedStateSize) {
    replaceDriver("/storage/emulated/0/Download/Turnip/Turnip-v22.3.1-R2/"); // <--- MODIFIED                       
    LOGV("Creating: %p", activity);
    activity->callbacks->onDestroy = onDestroy;
    // .. rest of the the code
}

However the result is the same.

I would appreciate some guidance on what am I missing or what am I doing wrong.

Thanks!

Matias

bylaws commented 7 months ago

Hi, double check you set useLegacyPackaging to true in your gradle config, this is required for it to pickup the hooks

darksylinc commented 7 months ago

Hi!

Thanks for the reply.

I just added useLegacyPackaging = true and set minSdk = 19 just in case (given [this ominous warning](https://developer.android.com/reference/tools/gradle-api/8.3/com/android/build/api/variant/JniLibsApkPackaging#useLegacyPackagingFromBundle()) that it's forced to false if API level >= 23; though I'm using APK, not bundles); but I'm afraid the problem remains.

bylaws commented 7 months ago

@darksylinc looked into this more, the arguments you pass to adrenotools_open_libvulkan are a bit off, and you aren't passing app libdir, see https://github.com/hrydgard/ppsspp/blob/768174e19ca80bccadc067bd2c7adc0d35fb4602/Common/GPU/Vulkan/VulkanLoader.cpp#L316, also the driver mustn't be on the sdcard and must be in the apps data directory.

darksylinc commented 7 months ago

YES!!!! YES YES YES!

THANK YOU!

I got it working!

First I had confused the /data/user/0/* folder with the /storage/emulated/0/* folder.

But more importantly, it is not clear at all that hookDir MUST point to whatever getApplicationInfo().nativeLibraryDir (Java code) says.

Once I tidy up some lose ends I will submit a PR to improve documentation on all the traps I fell while implementing this.

Thank you again!