yangxy112683 commented 7 months ago

背景

我们接入 Shadow 的 sdk 在升级了 targetSDKVersion到 30 之后，集成到项目中后，收到相关异常的反馈，查看都是发生在 Android 12 的机器上，以华为手机居多

初步看问题堆栈，定位问题是在PluginClassLoader创建 dex 时，想了解下对于高版本系统，Shadow 最新版本是否有过相关的兼容处理?

堆栈信息

#00 pc 0000000000089fcc /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) [arm64-v8a] 
#01 pc 000000000065d050 /apex/com.android.art/lib64/libart.so (art::Runtime::Abort(char const*)+740) [arm64-v8a] 
#02 pc 00000000000159e0 /system/lib64/libbase.so [arm64-v8a] 
#03 pc 0000000000015004 /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+484) [arm64-v8a] 
#04 pc 00000000006e2550 /apex/com.android.art/lib64/libart.so (art::VdexFile::GetNextTypeLookupTableData(unsigned char const*, unsigned int) const+152) [arm64-v8a] 
#05 pc 00000000005c5414 /apex/com.android.art/lib64/libart.so (art::OatFileBase::Setup(std::__1::vector> const&)+148) [arm64-v8a] 
#06 pc 00000000005c998c /apex/com.android.art/lib64/libart.so (art::OatFile::OpenFromVdex(int, std::__1::unique_ptr<art::vdexfile, std::__1::default_delete="std::__1::default_delete">>&&, std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator> const&, std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>*)+1204) [arm64-v8a] 
#07 pc 00000000005cfc3c /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::OatFileInfo::GetFile()+904) [arm64-v8a] 
#08 pc 00000000005cf14c /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::OatFileInfo::Status()+72) [arm64-v8a] 
#09 pc 00000000005cef30 /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::GetBestInfo(bool)+116) [arm64-v8a] 
#10 pc 00000000005d2d14 /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::GetOptimizationStatus(std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>*, std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>*, std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>*, std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>*)+64) [arm64-v8a] 
#11 pc 00000000005d41ac /apex/com.android.art/lib64/libart.so (art::OatFileManager::OpenDexFilesFromOat(char const*, _jobject*, _jobjectArray*, art::OatFile const**, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits="std::__1::char_traits">, std::__1::allocator>>>*)+616) [arm64-v8a] 
#12 pc 0000000000588658 /apex/com.android.art/lib64/libart.so (art::gc::Heap::CountInstances(std::__1::vector<art::Handle, std::__1::allocator<art::Handle>> const&, bool, unsigned long*)+600) [arm64-v8a] 
#13    pc 0000000000000870    /apex/com.android.art/javalib/arm64/boot-core-libart.oat [arm64-v8a]
java:
dalvik.system.DexFile.openDexFile(DexFile.java:371)
dalvik.system.DexFile.(DexFile.java:113)
dalvik.system.DexFile.(DexFile.java:86)
dalvik.system.DexPathList.loadDexFile(DexPathList.java:438)
dalvik.system.DexPathList.makeDexElements(DexPathList.java:397)
dalvik.system.DexPathList.(DexPathList.java:166)
dalvik.system.BaseDexClassLoader.(BaseDexClassLoader.java:134)
dalvik.system.BaseDexClassLoader.(BaseDexClassLoader.java:92)
com.tencent.estv.shadow.core.loader.classloaders.PluginClassLoader.(Unknown Source:10)
com.tencent.estv.shadow.core.loader.blocs.LoadApkBloc.loadPlugin(Unknown Source:265)
com.tencent.estv.shadow.core.loader.blocs.LoadPluginBloc$loadPlugin$buildClassLoader$1.call(Unknown Source:13)
com.tencent.estv.shadow.core.loader.blocs.LoadPluginBloc$loadPlugin$buildClassLoader$1.call(Unknown Source:0)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
java.lang.Thread.run(Thread.java:930)
</char,></char,></char,></char,></char,></char,></char,></char,></art::vdexfile,>

项目背景

为了防止 dex2oat造成插件内联优化，我们 sdk 会把 oat 文件夹删除

/**
 * 为了防止dex2oat造成插件内联优化，这里把oat文件夹删除
 * @param installedPlugin
 */
private void delOatDir(InstalledPlugin installedPlugin) {
    if(Build.VERSION.SDK_INT < Build.VERSION_CODES.P) return;

    try {
        File oatDir = new File(installedPlugin.runtimeFile.pluginFile.getParentFile(), "oat");
        if (oatDir.exists()) {
            MinFileUtils.cleanDirectory(oatDir);
        }
        Log.d(Constant.LOG_PREFIX + "aot", "oatDir del success:" + oatDir.getPath());
    } catch (Exception e) {
        Log.e(Constant.LOG_PREFIX + "aot", "oatDir del error:" + e.getMessage());
    }
}

目前项目中引入的 Shadow 版本不是最新版本，PluginCloassLoader 还是继承 DexClassLoader

class PluginClassLoader(
    dexPath: String,
    optimizedDirectory: File?,
    librarySearchPath: String?,
    parent: ClassLoader,
    private val specialClassLoader: ClassLoader?, hostWhiteList: Array<String>?
) : DexClassLoader(dexPath, optimizedDirectory?.path, librarySearchPath, parent) {

shifujun commented 7 months ago

你贴的堆栈只能说明你用的不是这个项目的原始代码。而你又不愿意分享你修改后的代码。所以这没有开源项目的交流前提。

即便如此。你想知道在你拿了代码修改之后，上游项目改了什么。你可以简单的用git diff看出来多出来哪些提交，那些文件有差异。如果这些提交的提交日志不清晰，如果文件的改动看不懂，都可以直接引用提issue。要求改进也是合理的。直接提PR改进代码可读性也是欢迎的。

dex和oat都属于Android的私有api，shadow从原理上不考虑介入。

yangxy112683 commented 7 months ago

你贴的堆栈只能说明你用的不是这个项目的原始代码。而你又不愿意分享你修改后的代码。所以这没有开源项目的交流前提。

即便如此。你想知道在你拿了代码修改之后，上游项目改了什么。你可以简单的用git diff看出来多出来哪些提交，那些文件有差异。如果这些提交的提交日志不清晰，如果文件的改动看不懂，都可以直接引用提issue。要求改进也是合理的。直接提PR改进代码可读性也是欢迎的。

dex和oat都属于Android的私有api，shadow从原理上不考虑介入。

这里并没有对 Shadow 的 loadPlugin 相关流程以及代码做过修改，只是为了解决项目中其他第三方组件也使用了 shadow，导致包名冲突，将 com.tencent.shadow 修改为 com.tencent.estv.shadow;

如果说的是删除 oat 的逻辑，目前只是在 Shadow 提供的示例 FastPluginManager的 loadPlugin()方法中，增加了删除oat 文件的操作，为了避免Android P 内联优化导致的问题，以下是相关代码；

    protected void loadPlugin(InstalledPlugin installedPlugin, String partKey, Boolean multiProcess) throws RemoteException, TimeoutException, FailedException {
        loadPluginLoaderAndRuntime(installedPlugin.UUID, partKey, multiProcess);
        Map map = mPluginLoader.getLoadedPlugin();
        if (!map.containsKey(partKey)) {
            delOatDir(installedPlugin);
            mPluginLoader.loadPlugin(partKey);
        }
    }

针对上游项目后续的改动，看到有针对 API 27 去掉 odex 的过程，这里会同步更新下相关改动尝试下。 API 26 保留 odex 过程

但该问题是在sdk 升级到 targetSDKVersion = 30 时才出现，不确认是否是否该问题导致，后续如还有发现的话，会将 shadow 上游改动同步到项目中

yangxy112683 commented 7 months ago

获取到更详细的崩溃堆栈，目前看起来是崩溃在runtime 中的 vdex_file.cc中的GetNextDexFileData()方法，该方法在 Android12中引入, 在 Android12 版本的源码中，有CHECK_ALIGN(data,4)的检查

该 check 在 Android13 版本中去掉了

estPlugi: vdex_file.cc:195] Check failed: ::art::IsAligned<4>(data) 0x7d4f902246
estPlugi: runtime.cc:689] Runtime aborting...
estPlugi: runtime.cc:689] Dumping all threads without mutator lock held
estPlugi: runtime.cc:689] All threads:
estPlugi: runtime.cc:689] DALVIK THREADS (42):
estPlugi: runtime.cc:689] "pool-7-thread-1" prio=5 tid=33 Runnable
estPlugi: runtime.cc:689] | group="" sCount=0 ucsCount=0 flags=0 obj=0x1329d3a8 self=0xb400007bd05e7000
estPlugi: runtime.cc:689] | sysTid=31136 nice=0 cgrp=foreground sched=0/0 handle=0x7bc2095cb0
estPlugi: runtime.cc:689] | state=R schedstat=( 39925522 1402602 21 ) utm=2 stm=1 core=6 HZ=100
estPlugi: runtime.cc:689] | stack=0x7bc1f92000-0x7bc1f94000 stackSize=1039KB
estPlugi: runtime.cc:689] | held mutexes= "abort lock" "mutator lock"(shared held)
estPlugi: runtime.cc:689] native: #00 pc 00000000005842b4 /apex/com.android.art/lib64/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, int, BacktraceMap*, char const*, art::ArtMethod*, void*, bool)+140)
estPlugi: runtime.cc:689] native: #01 pc 00000000006b3b58 /apex/com.android.art/lib64/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, bool, BacktraceMap*, bool) const+364)
estPlugi: runtime.cc:689] native: #02 pc 00000000006d20d8 /apex/com.android.art/lib64/libart.so (art::DumpCheckpoint::Run(art::Thread*)+944)
estPlugi: runtime.cc:689] native: #03 pc 00000000006cb964 /apex/com.android.art/lib64/libart.so (art::ThreadList::RunCheckpoint(art::Closure*, art::Closure*)+564)
estPlugi: runtime.cc:689] native: #04 pc 00000000006cab10 /apex/com.android.art/lib64/libart.so (art::ThreadList::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, bool)+1504)
estPlugi: runtime.cc:689] native: #05 pc 000000000065d62c /apex/com.android.art/lib64/libart.so (art::Runtime::Abort(char const*)+2240)
estPlugi: runtime.cc:689] native: #06 pc 00000000000159e0 /system/lib64/libbase.so (android::base::SetAborter(std::__1::function<void (char const*)>&&)::$_3::__invoke(char const*)+76)
estPlugi: runtime.cc:689] native: #07 pc 0000000000015004 /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+484)
estPlugi: runtime.cc:689] native: #08 pc 00000000006e2550 /apex/com.android.art/lib64/libart.so (art::VdexFile::GetNextTypeLookupTableData(unsigned char const*, unsigned int) const+152)
estPlugi: runtime.cc:689] native: #09 pc 00000000005c5414 /apex/com.android.art/lib64/libart.so (art::OatFileBase::Setup(std::__1::vector<art::DexFile const*, std::__1::allocator<art::DexFile const*> > const&)+148)
estPlugi: runtime.cc:689] native: #10 pc 00000000005c998c /apex/com.android.art/lib64/libart.so (art::OatFile::OpenFromVdex(int, std::__1::unique_ptr<art::VdexFile, std::__1::default_delete<art::VdexFile> >&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*)+1204)
estPlugi: runtime.cc:689] native: #11 pc 00000000005cfc3c /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::OatFileInfo::GetFile()+904)
estPlugi: runtime.cc:689] native: #12 pc 00000000005cf14c /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::OatFileInfo::Status()+72)
estPlugi: runtime.cc:689] native: #13 pc 00000000005cef30 /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::GetBestInfo(bool)+116)
estPlugi: runtime.cc:689] native: #14 pc 00000000005d2d14 /apex/com.android.art/lib64/libart.so (art::OatFileAssistant::GetOptimizationStatus(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*)+64)
estPlugi: runtime.cc:689] native: #15 pc 00000000005d41ac /apex/com.android.art/lib64/libart.so (art::OatFileManager::OpenDexFilesFromOat(char const*, _jobject*, _jobjectArray*, art::OatFile const**, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >*)+616)
estPlugi: runtime.cc:689] native: #16 pc 0000000000588658 /apex/com.android.art/lib64/libart.so (art::DexFile_openDexFileNative(_JNIEnv*, _jclass*, _jstring*, _jstring*, int, _jobject*, _jobjectArray*)+144)
estPlugi: runtime.cc:689] at dalvik.system.DexFile.openDexFileNative(Native method)
estPlugi: runtime.cc:689] at dalvik.system.DexFile.openDexFile(DexFile.java:371)
estPlugi: runtime.cc:689] at dalvik.system.DexFile.<init>(DexFile.java:113)
estPlugi: runtime.cc:689] at dalvik.system.DexFile.<init>(DexFile.java:86)
estPlugi: runtime.cc:689] at dalvik.system.DexPathList.loadDexFile(DexPathList.java:438)
estPlugi: runtime.cc:689] at dalvik.system.DexPathList.makeDexElements(DexPathList.java:397)
estPlugi: runtime.cc:689] at dalvik.system.DexPathList.<init>(DexPathList.java:166)
estPlugi: runtime.cc:689] at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:134)
estPlugi: runtime.cc:689] at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:92)
estPlugi: runtime.cc:689] at com.tencent.estv.shadow.core.loader.classloaders.PluginClassLoader.<init>(unavailable:-1)
estPlugi: runtime.cc:689] at com.tencent.estv.shadow.core.loader.blocs.LoadApkBloc.loadPlugin(unavailable:-1)
estPlugi: runtime.cc:689] at com.tencent.estv.shadow.core.loader.blocs.LoadPluginBloc$loadPlugin$buildClassLoader$1.call(unavailable:-1)
estPlugi: runtime.cc:689] at com.tencent.estv.shadow.core.loader.blocs.LoadPluginBloc$loadPlugin$buildClassLoader$1.call(unavailable:-1)
estPlugi: runtime.cc:689] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
estPlugi: runtime.cc:689] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
estPlugi: runtime.cc:689] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
estPlugi: runtime.cc:689] at java.lang.Thread.run(Thread.java:930)
estPlugi: runtime.cc:689]

Tencent / Shadow

Android12(API = 31) 在 loadPlugin 时出现Runtime::Abort, 猜测异常与 dex 创建有关 #1264

背景

堆栈信息

项目背景