yuenshome / yuenshome.github.io

https://yuenshome.github.io
MIT License
81 stars 15 forks source link

软件发布前的优化与裁剪:发布包大小与逆向(上) #92

Open ysh329 opened 4 years ago

ysh329 commented 4 years ago

先发两本书关于iOS和Android开发,可能后续会参考其中内容:


ysh329 commented 4 years ago

在安卓或iOS部署APP过程中,APP本身就要有不少资源占了体积,尤其是一些APP的图片素材等。还要考虑把端侧推理引擎引入,自然会考虑这部分带来的增量:一个或者多个模型的体积,包含一个或者多个模型对应算子的一个或者多个(注:不同团队用不同的引擎)推理引擎包的体积。

既然说APP体积,那么就先以安卓来说明,看下APK来自wiki的定义:

Android应用程序包(英语:Android application package,APK),是Android操作系统使用的一种应用程序包文件格式,用于分发和安装移动应用及中间件。

  • 一个Android应用程序的代码想要在Android设备上运行,必须先进行编译,然后被打包成为一个被Android系统所能识别的文件才可以被运行,而这种能被Android系统识别并运行的文件格式便是“APK”。
  • 一个APK文件内包含被编译的代码文件(.dex 文件),文件资源(resources), assets,证书(certificates),和清单文件(manifest file)。
ysh329 commented 4 years ago

1. 安卓APK的构成

前面已经简单介绍了APK,后面会深入来看,不过先来明确要讨论的问题:

  1. Android APP上体积是怎么看的,宏观层面;
  2. 当增加内容如一个lib时如何估计引入的增量;
  3. 目前安卓上压缩体积的常用方法(略)。

根据Android官方文档对使用 APK 分析器分析您的编译版本这一节的说明显示,APK是压缩格式,符合zip标准。目前有两种方式查看其中的内容:

image 方式二:通过menu中的Build菜单找到apk源文件

1.1 安卓APK的体积:抖音为例

下面以抖音为例,从抖音官网下载APK包,用第一种方法,在Mac上用下图中的方式解压:

image
方式一:解压查看apk内容

第二种方法,需要将APK文件拖到Android Studio里,并双击可看到如下内容,能看到4个字段,分别是:File、Raw File Size、Download Size、% of Total Download Size。其中解释一下后3个字段:

image

该APK体积占比前4的部分,占了总体积的95%左右,既然分析体积,那分别来看一下分别是什么:

  1. lib(占37.6%):包含抖音APP所用到的动态库(具体见下面第二张图),仅包含armeabi-v7a这个文件夹,应该是为了低端机兼容性的考虑;
  2. r(占13.7%):包含超多的文件夹,文件夹和文件的名称都用了混淆并减小体积,文件以xml、png、webp为主,看似是不少APP内素材图片和对应设置。APK包根目录下包含一个名为resources.arsc的文件,是编译后的二进制资源文件,包括图片、文本索引等,虽然是二进制但内容可以通过Android Studio里查看,其中文件夹r猜测是res的简写,该文件与本文件夹内的如xml等文件资源做了映射;
    image
  3. assets(占7.3%):包括了所有的图片,raw 文件和 XML。当然,也包含了模型,assets/model/下面有两个人脸相关的模型文件夹:ttfacemodelttfaceattrmodel;此外散落在assets下的三个(疑似)模型:arch1.netarch2.netarch3.net
  4. dex文件(7个classes.dex文件,加起来占23.6MB,37%跟lib差不多):dex文件作为Android APK的组成部分,其主要是Android的Java代码经过编译生成class文件,再经过dx命令生成的。Android应用是运行在虚拟机上的,工程编译过程中会把java文件编译成class,再转换成dex文件。若method或field超过了65k,为了保证在dialvik上能运行,还需引入multidex分包,如上图中就有多个dex文件。 image
    下面是Android官方文档对Dex文件的说明,结合上图的列名,一起来学习一下吧:

    Dex文件包含字段

    • Defined Methods 和 Referenced Methods 列:列出了 DEX 文件中每个软件包、类和方法的相关计数;
    • Referenced Methods列:统计由 DEX 文件引用的所有方法。这通常包括在您的代码和依赖项库中定义的方法,以及在代码使用的标准 Java 和 Android 软件包中定义的方法 - 这些方法计入每个 DEX 文件中的 64k 方法限制(补充:所以能看到不少apk包解压后,有多个dex文件);
    • Defined Methods 列:仅统计在某一个 DEX 文件中定义的方法,所以此数字是 Referenced Methods 的子集。
      注:如果将某个依赖项打包在 APK 中,这两个方法计数都会加上在该依赖项中定义的方法。另请注意,缩减大小和 Proguard 压缩功能也会在源代码经过编译后大大改变 DEX 文件的内容(补充Proguard:ProGuard是一个压缩、优化和混淆Java字节码文件的免费的工具,它可以删除无用的类、字段、方法和属性。可以删除没用的注释,最大限度地优化字节码文件。它还可以使用简短的无意义的名称来重命名已经存在的类、字段、方法和属性。常常用于Android开发用于混淆最终的项目,增加项目被反编译的难度)。

这里点到为止,后面会再提到。

1.2 引擎库、模型和Java调用源码

因为比较关注ai推理这块,那就自然包括了库、模型文件、调用代码(dex文件)。下面分别讲一下在这三个上的小发现。

1.2.1 引擎库

armeabi-v7a文件夹下有不少动态库,其中容易辨识的是体积为224.5KB的libc++_shared.so,在ndk中如我的android-ndk-r17c中搜了下,根据平台可以确定是下面这个:

# armeabi-v7a的libc++_shared.so的体积
644K    ./android-ndk-r17c/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++_shared.so

# file命令查看该libc++_shared.so的基本信息
./android-ndk-r17c/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++_shared.so: 
# ELF 32-bit LSB shared object,
# ARM, EABI5 version 1 (SYSV), 
# dynamically linked, 
# BuildID[sha1]=062c84f6da90c5346e6d26986fa578e8f3c7ffd2,
# stripped

从体积上来ndk里看是644KB,而且已被stripped了,apk包里的Raw File Size和Download Size那两项,分别仅有230~240KB左右,这两项分别代表什么,难道Android Studio又做了裁剪么?

根据Android官方文档对查看文件和大小信息 | Android Studio 用户指南,了解到:

那看来Android Studio并没做裁剪,那常识对libc++_shared.so用zip命令实验一下:

$ zip libc++_shared.so.zip ./android-ndk-r17c/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++_shared.so
  adding: android-ndk-r17c/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++_shared.so (deflated 64%)
$ du -sh ./libc++_shared.so.zip
236K    ./libc++_shared.so.zip

对libc++_shared.so做zip默认级别的压缩,从644KB直降到236KB,虽然不清楚这个APP开发者使用的具体NDK版本(15/16/17,Linux/Mac),不过已经与APK中看到的libc++_shared.so的Download Size这一项的235.4KB基本一致。

整个armeabi-v7a下的所有so包,解压后的整体体积为54MB,zip压缩后为24MB。每个so包的压缩比在zip命令执行过程中就可以看到。

综上,引入动态库对APK体积的增量,可通过对其用zip命令,观察前后变化查看;有的静态库会打入动态库中,体积增量以zip后的动态库为准。


1.2.2 真正的引擎库

直接在lib/armeabi-v7a目录下使用grep命令检索关键词:Conv、Convolution、face_detect、Beauty、beauty、inference、predict等,最后定位到以下3个so:

注:根据目录名片段,可检索到一大把姓名拼音缩写,以及对应模块代码,若运气好在微信里以拼音可检索到对应人名(含未加好友的群成员名的拼音),核对信息大概率是这个人,或许我可以挖人了。

libsrlib

对libsrlib.so使用vim命令直接打开观察,再配合stings、nm、grep等命令组合,检索到有用到MNN引擎,以及shrinkMNN关键词,猜测对MNN压缩删除未使用到的代码。

并在其中找到几个文件名:1.net、2.net、3.net,对应assets/目录下的三个文件(arch1.net、arch2.net、arch3.net),更多见后文。

libIESNN_OCL

对libIESNN_OCL.so使用stings、nm、grep等命令组合,可以确认推理引擎名为IESNN,CPU-armv7a支持14个CPU层和4个GPU(OpenCL)层,因为包含Input层,而且函数名带有Layer,估计最早支持的应该也是Caffe。

ysh329 commented 4 years ago

1.2.3 模型文件

在assets文件夹中,看到:

image

不排除还有其他模型,后面也以上面两组模型做分析,首先得到下面几个结论:

  1. 这2组模型中部分模型加密过。看不到模型结构信息,也看不到模型参数,第一组4个模型中,每个文件打开能看到对应模型名称信息,不过也根据strings(查找object或binary中的可打印字符串)以及grep(检索文件内包含指定pattern的内容)命令发现:

    • ttfaceattrmodel/tt_beauty_attr6_v1.0.model:该文件能看到第一行有一个tt_beauty_attr6_v1.0名字,后续或许可以通过该名字反查到所应用的dex或so包;
    • ttfaceattrmodel/tt_face_attribute_v5.0.model:同上,这次的关键词是v5.0agenet
    • ttfacemodel/tt_face_v7.0.model:第1行,v7.0detect,第1078行,第1511行,poutbase120det,第1593行,eyecls,第1678行,partface
    • ttfacemodel/tt_face_extra_v10.0.model:第1行很奇怪,第2行包含了v10.0iris,第54行包含extra,第2036行包含fastextra
  2. 用MNN作为部分模型的推理引擎。第2组的3个模型文件,直接vim打开,可看到MNNarch8/resize/ResizeBilinear___tr4arch8arch8/Conv/Conv2D等字样。不妨试试看能否用MNN加载,从而分析里面结构。
    第一次使用MNN例行先跑benchmark,去官网拉release代码,将benchmark目录下的模型文件下载到本地,如仅有28KB的resnet-v2-50.mnn,benchmark成功跑通(题外话:benchmark的文档竟然不告知用户需要打开根目录下benchmark的编译选项,一脸懵没找到benchmark.out这里可以改进加强一下)。再观察这3个模型文件都只有10KB上下,应该是只有结构不含参数。
    尝试用MNN的benchmark加载这3个模型,Ubuntu x86平台编译benchmark加载3个模型失败,Android平台编译benchmark加载3个模型成功跑通前2个(855、cpuv7-4threads、5ms、13ms,虽然模型大体结构一样,但是时间不同),根据MNN文档可用netron可视化前两个模型(需把后缀名改为.mnn),前两个模型结构一样如下图,可以看到filter大小,但是看不到输入输出大小【TODO】: image

    • 根据在lib/armeabi-v7a目录下检索这三个模型的名字,发现libsrlib.so调用了1.net、2.net、3.net这3个模型; 注:第三个加载segmentationFault在读取模型的地方,位于 Interpreter::createFromBufferInternal的地方,具体说就是net->net->oplists()为空,可能做了op的特别处理导致无法读取到等等,但即使如此,也能直接看到该文件中包含了模型input和output的名字:Placeholderarch8/xynout。后者的xynout不知道是个什么东西,在github上搜了一圈最终指向某人脸数据集的BioID_0285.pgm文件包含这一段字符,这个方向应该不正确。

1.2.3 Java调用代码(Dex文件)

模型文件就先看到这里。根据模型文件中拿到的关键词,接下来看看dex文件。首先将7个dex文件转为jar包,再转为java代码。网上不少反编译头条的文章,照着做了下,用jd-gui(这个项目github上可以搜到用release版即可)会有导出jar包卡死的问题,坑就是第一个dex文件会有解析问题,反编译后面的所有dex文件。

除了使用jd-core,还可以用dex2jar这个工具,也是需要啥在github上搜就行了,但是完整对apk导出的时候也会有问题:OutOfMemoryError: Java heap space(在更大内存的机器上试了,加了-XX:-UseGCOverheadLimit,均以失败告终),找到dex2jar对应源码,对apk解压出的每个dex文件,挨个dex导出即可,总之是把能导出的都导出了。

有点烦,不想写了就到这里吧,可以确定到哪个dex包,或者说哪个jar包调用了1.net、2.net、3.net,可以找到对应的Java代码,没啥意思,大家都早点睡吧。

ysh329 commented 4 years ago

image image

那么,抖音的AI推理引擎是哪个so呢(也有可能是.a,然后被打到某个.so中)?从体积上看,先排除掉100KB以下的,根据上面的所说的zip压缩来看,这里的体积实际是被zip压缩过的大小。

1.2.3 调用代码(Dex文件)

【【【【【【【【】】】】】】】】 而说到资源必然要说的是图片优化,毕竟图片是资源体积中占大头。

我们可以选择体积更小的图片,比方说webp格式。要注意的是,webp在4.3以下有一些兼容问题,选择需谨慎。而对于png也可以采用有损或无损压缩图片,常见工具有tingpng或imageoptim等。经常png会有一些空白边距,这些部分也可以删除掉,通过代码或xml来实现边距。

小的icon也可以使用svg格式。一些特定效果也可以通过XML自定义实现。新手常常有个误区,就是做.9图片的原图依旧很大,既然已经让程序帮我们拉伸了,原图实际上可以保持很小的体积。

经常在适配屏幕时,开发会使用多套图片。实际上大部分时候一套图片就可以搞定了,系统会自动帮我们处理图片缩放。

而在适配不同API时,打包过程也会自动生成多个目录。适当选择最低API格式有好处的,更高版本意味着开发效率更高,同时兼容设备变少,需要同业务一起考虑。

打包过程也会生成一个resource.arsc文件,里面是资源ID的映射关系。这个部分也可以通过混淆资源id来优化,可以有效减少体积。

最后同样的,不需要的资源一定要删掉。同样使用lint工具可以帮我们自动删除。模块化也需要当心不同模块出现同样的图片。

【【【【【【dex】】】】】】

对于dex文件常见的优化策略:无用代码删除,IDE会把未使用到的区块灰色展示,也可以通过lint工具来帮助我们识别到。这个功能在menu中的Analyze的Run inpection by name输入unused xx。甚至是在编译过程中,我们也可以通过混淆配置来做到,比方说常见的Log删除。

而随着业务发展,大型app都进行了模块化组件化,不同团队负责开发不同模块。开发过程极易出现一些重复工具方法,甚至引入相同功能的不同组件。架构上也需要一下规约工具来限定重复代码的编写。

在class转换成dex过程中,我们可以用新一代d8工具,它可以帮助我们的dex体积更小运行速度更快。Facebook也开源了一款redex的工具帮助优化dex。 这里面包括了APK的源码,反编译时最主要就是对这个文件进行反编译。有人会问,知道了dex的文件结构有什么用呢?在Android安全方面来说,dex是安全的重头戏,如果能够了解了dex文件的格式,那么对于dex文件的加固原理也就有简单的了解。

image

其它参考:

image

lib

进到lib看,所有lib都是strip过的:

$ file ./*
./libAkeva.so:                      ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=5d5b33c1f4a20b1e3f09a73cb2118ba5022e9914, stripped
./libCtaApiLib.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=50aa95b697b49dbdd5d0476784c4f151d1903196, stripped
./libIESNN_OCL.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=3431978b81f8f7504779a74be9f77e1db344cdce, stripped
./libNN265dec.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=064390d39f5adb5e3a9c46252b66344fcba15961, stripped
./libalog-lib.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=5ced79e114ac9f0341cb8a78cba209eee21638c2, stripped
./libaudioeffect.so:                ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=f2dff4457cbb95b9e2fff07bcf97a89d4bd1d524, stripped
./libavmdl.so:                      ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=4fd998221e7c5c61cf3a97f2d4a3c584e4edfc3b, stripped
./libboost_multidex.so:             ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=a08ed2014ed96abdc78dd3bb4912222ed1f68844, stripped
./libbspatch.so:                    ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=e3570db7e63a2d75543347e8b150eb5bc85eb4d0, stripped
./libc++_shared.so:                 ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=006aa3f3cdc1d618f277596e399816ccdc765674, stripped
./libcms.so:                        ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=325765685bb6debd93035569d9a342e2ec8a63e7, stripped
./libcocklogic-1.1.3.so:            ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libcrashlytics.so:                ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=e3c978ad09d1fab2d6c86de96df95c6442868a1a, stripped
./libdaemon.so:                     ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=c283cd697270799171fb791d81d113ce08267a02, stripped
./libdexposed.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, stripped
./libdownloadable_model_support.so: ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=c6e64ebc9f67a9c35225f6e2005debf5d0a08c66, stripped
./libeffect.so:                     ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=f74fa697272484710a13ef6beb5e557be1da280e, stripped
./libfdk-aac.so:                    ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libflock-lib.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=4b078d92d333b0cf1ff1e33d584c3d4b13645cf2, stripped
./libgcsuppression.so:              ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=82430d75c3474edc17ac48ca5bf918f2422565f0, stripped
./libgifimage.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=c689e22d35c16d9a0e4e19ee8d8af64bc440738b, stripped
./libgraphicbuf.so:                 ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=21404c6a993f13fa5d9dcf7718a32df5d67a8c8a, stripped
./libhardwarebuf.so:                ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=4065052fbeda57c7ea5bc9f10261b98be9a1a23d, stripped
./libimagepipeline.so:              ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=8504d391ec61e28d9e4617c6e5c7577582280752, stripped
./libnative-filters.so:             ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=25f6af623a5960cd367eb6aa5d0401936dafb145, stripped
./libneverland.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=d4979a1d7244ed699830f861989ffcb1958d8243, stripped
./libnpth.so:                       ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=f1d161abcb842977a685fda8b87df663b11a84b3, stripped
./libnpth_dumper.so:                ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=d81d9aef41a6cae314b56df8a05b1d27211f0ed4, stripped
./libpl_droidsonroids_gif.so:       ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=29c7de0cc7be5969cd4dca0dc0a7cdf5b7a1b0b1, stripped
./libreactnativejni.so:             ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=6388caf2caf4c22e85e22cce067da51bf8db2f64, stripped
./libsrlib.so:                      ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=b95e77f21094eb896cc2865b92f93079b470d5c6, stripped
./libsscronet.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=3211639b33e9a00e1d94b40a1c54e819ba57e987, stripped
./libstatic-webp.so:                ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=a996b1897b17f176737c1d4d2f224a08c270fe8e, stripped
./libsupervisor.so:                 ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=713e3bddee92ba671fe9172ea62f6fe3dc4c0db0, stripped
./libsysoptimizer.so:               ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, stripped
./libtailor.so:                     ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=985c55d7b2f1f0dd83d083769b80cceeda04ba52, stripped
./libtnet-3.1.14.so:                ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=cc637679049c0eb55c2846bf18e7daaacb9f6645, stripped
./libttEncrypt.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libttffmpeg.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libttmain.so:                     ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=f4ab695c4ca488fb74e9bd7f73250c6b0f3cf0c7, stripped
./libttmplayer.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=8207fe5d3692dfb70dc5a8eef211c1d1e70bc7ee, stripped
./libttopenssl.so:                  ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libttvebase.so:                   ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=ba3c5b9d9b3f14ba16db356bd78be39d8c84ca72, stripped
./libttvideoeditor.so:              ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=7784416fb18b7fb445aca9d3c17c48556f7513a2, stripped
./libttvideorecorder.so:            ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=7d2ec40e12f7c0535630eff42cfbcdc5c39d574e, stripped
./libttvideouploader.so:            ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libv8.cr.so:                      ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, stripped
./libv8_libbase.cr.so:              ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, stripped
./libv8_libplatform.cr.so:          ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, stripped
./libwcdb.so:                       ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[sha1]=4a447821d940a657620c2a944011cc5761e5ac7a, stripped
./libweibosdkcore.so:               ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, stripped
./libx264.so:                       ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
./libyuv.so:                        ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=9748de7ab8b8dae4c36e179dddb7e7282ba6acb8, stripped
$ grep -rn infer .
Binary file ./libeffect.so matches
Binary file ./libsrlib.so matches
Binary file ./libttffmpeg.so matches
Binary file ./libv8.cr.so matches
$ grep -rn beauty .
Binary file ./libeffect.so matches
Binary file ./libttvideorecorder.so matches
Binary file ./libsscronet.so matches
Binary file ./libttvideoeditor.so matches
Binary file ./libttvebase.so matches
$ grep -rn face_attr .
Binary file ./libeffect.so matches
Binary file ./libttvideorecorder.so matches
Binary file ./libttvebase.so matches

$ du -sh ./libeffect.so ./libttvebase.so ./libttvideorecorder.so
 19M    ./libeffect.so
1.5M    ./libttvebase.so
928K    ./libttvideorecorder.so

$ nm -D ./libttvideorecorder.so | grep face
         U ANativeWindow_fromSurface
00071df0 T Java_com_ss_android_medialib_FaceBeautyInvoker_nativeChangeSurface
000735fa T Java_com_ss_android_medialib_FaceBeautyInvoker_nativeInitMediaCodecSurface
000ae4b0 T _ZN11GLEnvHelper15getSurfaceWidthEv
000ae4ac T _ZN11GLEnvHelper16getSurfaceHeightEv
0003a80c T _ZN12TextureUtils17genSurfaceTextureEv
00039f78 T _ZN15SharedGLContext13createSurfaceENS_14EEGLConfigTypeEP13ANativeWindow
         U _ZN22TEStickerEffectWrapper20getSmartBeautyResultEP24bef_face_attrs_result_st
         U _ZN22TEStickerEffectWrapper22getFaceDetectResultEffEP16bef_face_info_st
         U _ZN22TEStickerEffectWrapper22initFaceDetectExtParamERK28bef_face_detect_ext_param_st
         U _ZN22TEStickerEffectWrapper28initFaceBeautyDetectExtParamERK35bef_face_beauty_detect_ext_param_st
         U _ZN22TEStickerEffectWrapper30initFaceBeautifyDetectExtParamERK37bef_face_beautify_detect_ext_param_st
         U _ZNKSt6__ndk16locale9has_facetERNS0_2idE
         U _ZNKSt6__ndk16locale9use_facetERNS0_2idE
         U bef_effect_photo_face_detect_and_clip
         U bef_effect_photo_face_detect_clear_textures
         U bef_effect_photo_face_detect_destroy
         U bef_effect_photo_face_detect_filter_policy
         U bef_effect_photo_face_detect_pick_resources
         U bef_effect_photo_face_detect_setparam
         U eglCreatePbufferSurface
         U eglCreatePixmapSurface
         U eglCreateWindowSurface
         U eglDestroySurface
         U eglQuerySurface

$ nm -D ./* | grep face_detect
0016e75e T bef_effect_face_detect
0016e568 T bef_effect_face_detect_create
0016e79e T bef_effect_face_detect_setparam
00171ea8 T bef_effect_get_face_detect_result
00163e9c T bef_effect_photo_face_detect
00163f24 T bef_effect_photo_face_detect_and_clip
00163f30 T bef_effect_photo_face_detect_clear_textures
00163d04 T bef_effect_photo_face_detect_create
00163ec0 T bef_effect_photo_face_detect_destroy
00163ed4 T bef_effect_photo_face_detect_filter_policy
00163e54 T bef_effect_photo_face_detect_pick_resources
00163f10 T bef_effect_photo_face_detect_set_output_size
00163ea8 T bef_effect_photo_face_detect_setparam
0005d5b4 T _ZN22TEStickerEffectWrapper22initFaceDetectExtParamERK28bef_face_detect_ext_param_st
         U bef_effect_get_face_detect_result
         U bef_effect_get_face_detect_result
         U _ZN22TEStickerEffectWrapper22initFaceDetectExtParamERK28bef_face_detect_ext_param_st
         U bef_effect_photo_face_detect_and_clip
         U bef_effect_photo_face_detect_clear_textures
         U bef_effect_photo_face_detect_destroy
         U bef_effect_photo_face_detect_filter_policy
         U bef_effect_photo_face_detect_pick_resources
         U bef_effect_photo_face_detect_setparam
ysh329 commented 4 years ago

taobao

lib / armeabi-v7a / $ grep -rn inference .
Binary file ./libalinnface-v7a.so matches
Binary file ./libtbffmpeg.so matches
Binary file ./libclasses.so matches

lib / armeabi-v7a / $ nm -D ./libalinnface-v7a.so | grep " T "
0000aa5c T DeinitFaceModule
0000a958 T DetectFaceAndLandmark
0000a8d0 T InitFaceModule
00002ba4 T JNI_OnLoad
0000a928 T SetSizeAndAngle
ysh329 commented 4 years ago

dex2jar

Releases · pxb1988/dex2jar https://github.com/pxb1988/dex2jar/releases

在首页readme查看如何使用:

sh d2j-dex2jar.sh -f ~/path/to/apk_to_decompile.apk

# And the output file will be apk_to_decompile-dex2jar.jar.

下载release版本,并设置对应的环境变量到~/.bashrc

$ d2j-dex2jar.sh -f aweme_aweGW_v8.8.0_55fb587.apk
dex2jar aweme_aweGW_v8.8.0_55fb587.apk -> ./aweme_aweGW_v8.8.0_55fb587-dex2jar.jar
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3664)
        at java.lang.String.<init>(String.java:207)
        at java.lang.StringBuilder.toString(StringBuilder.java:407)
        at com.googlecode.d2j.util.Mutf8.decode(Mutf8.java:40)
        at com.googlecode.d2j.reader.DexFileReader.getString(DexFileReader.java:965)
        at com.googlecode.d2j.reader.DexFileReader.getType(DexFileReader.java:975)
        at com.googlecode.d2j.reader.DexFileReader.getField(DexFileReader.java:917)
        at com.googlecode.d2j.reader.DexFileReader.acceptInsn(DexFileReader.java:1595)
        at com.googlecode.d2j.reader.DexFileReader.acceptCode(DexFileReader.java:1407)
        at com.googlecode.d2j.reader.DexFileReader.acceptMethod(DexFileReader.java:1058)
        at com.googlecode.d2j.reader.DexFileReader.acceptClass(DexFileReader.java:849)
        at com.googlecode.d2j.reader.DexFileReader.accept(DexFileReader.java:659)
        at com.googlecode.d2j.reader.MultiDexFileReader.accept(MultiDexFileReader.java:116)
        at com.googlecode.d2j.reader.MultiDexFileReader.accept(MultiDexFileReader.java:109)
        at com.googlecode.d2j.dex.Dex2jar.doTranslate(Dex2jar.java:88)
        at com.googlecode.d2j.dex.Dex2jar.to(Dex2jar.java:275)
        at com.googlecode.dex2jar.tools.Dex2jarCmd.doCommandLine(Dex2jarCmd.java:112)
        at com.googlecode.dex2jar.tools.BaseCmd.doMain(BaseCmd.java:290)
        at com.googlecode.dex2jar.tools.Dex2jarCmd.main(Dex2jarCmd.java:33)

在issue区搜索报错关键词OutOfMemoryError: Java heap space找到一个关于windwos平台上的报错,有一个回复是告知提问者修改某个.bat文件中设置的,那我mac平台,就需改同名.sh文件,反正两个文件都改了还是有这个问题,又在超大内存的开发机上试试:

# ./dex-tools-2.1-SNAPSHOT/d2j-dex2jar.sh -f ./aweme_aweGW_v8.8.0_55fb587.apk
dex2jar ./aweme_aweGW_v8.8.0_55fb587.apk -> ./aweme_aweGW_v8.8.0_55fb587-dex2jar.jar
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
    at java.lang.StringBuilder.<init>(StringBuilder.java:101)
    at com.googlecode.d2j.reader.DexFileReader.getString(DexFileReader.java:964)
    at com.googlecode.d2j.reader.DexFileReader.getType(DexFileReader.java:975)
    at com.googlecode.d2j.reader.DexFileReader.getField(DexFileReader.java:917)
    at com.googlecode.d2j.reader.DexFileReader.acceptInsn(DexFileReader.java:1595)
    at com.googlecode.d2j.reader.DexFileReader.acceptCode(DexFileReader.java:1407)
    at com.googlecode.d2j.reader.DexFileReader.acceptMethod(DexFileReader.java:1058)
    at com.googlecode.d2j.reader.DexFileReader.acceptClass(DexFileReader.java:856)
    at com.googlecode.d2j.reader.DexFileReader.accept(DexFileReader.java:659)
    at com.googlecode.d2j.reader.MultiDexFileReader.accept(MultiDexFileReader.java:116)
    at com.googlecode.d2j.reader.MultiDexFileReader.accept(MultiDexFileReader.java:109)
    at com.googlecode.d2j.dex.Dex2jar.doTranslate(Dex2jar.java:88)
    at com.googlecode.d2j.dex.Dex2jar.to(Dex2jar.java:275)
    at com.googlecode.dex2jar.tools.Dex2jarCmd.doCommandLine(Dex2jarCmd.java:112)
    at com.googlecode.dex2jar.tools.BaseCmd.doMain(BaseCmd.java:290)
    at com.googlecode.dex2jar.tools.Dex2jarCmd.main(Dex2jarCmd.java:33)

看到同样问题:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded,不过也是无解状态。

ysh329 commented 4 years ago

java.lang.OutOfMemoryError:GC overhead limit exceeded解决方法 - andybbc的博客 - CSDN博客 https://blog.csdn.net/andybbc/article/details/50581717

尝试,在其中加入

-XX:-UseGCOverheadLimit

# ./dex-tools-2.1-SNAPSHOT/d2j-dex2jar.sh -f ./aweme_aweGW_v8.8.0_55fb587.apk
dex2jar ./aweme_aweGW_v8.8.0_55fb587.apk -> ./aweme_aweGW_v8.8.0_55fb587-dex2jar.jar
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
    at java.lang.StringBuilder.<init>(StringBuilder.java:101)
    at com.googlecode.d2j.reader.DexFileReader.getString(DexFileReader.java:964)
    at com.googlecode.d2j.reader.DexFileReader.getType(DexFileReader.java:975)
    at com.googlecode.d2j.reader.DexFileReader.getField(DexFileReader.java:917)
    at com.googlecode.d2j.reader.DexFileReader.acceptInsn(DexFileReader.java:1595)
    at com.googlecode.d2j.reader.DexFileReader.acceptCode(DexFileReader.java:1407)
    at com.googlecode.d2j.reader.DexFileReader.acceptMethod(DexFileReader.java:1058)
    at com.googlecode.d2j.reader.DexFileReader.acceptClass(DexFileReader.java:849)
    at com.googlecode.d2j.reader.DexFileReader.accept(DexFileReader.java:659)
    at com.googlecode.d2j.reader.MultiDexFileReader.accept(MultiDexFileReader.java:116)
    at com.googlecode.d2j.reader.MultiDexFileReader.accept(MultiDexFileReader.java:109)
    at com.googlecode.d2j.dex.Dex2jar.doTranslate(Dex2jar.java:88)
    at com.googlecode.d2j.dex.Dex2jar.to(Dex2jar.java:275)
    at com.googlecode.dex2jar.tools.Dex2jarCmd.doCommandLine(Dex2jarCmd.java:112)
    at com.googlecode.dex2jar.tools.BaseCmd.doMain(BaseCmd.java:290)
    at com.googlecode.dex2jar.tools.Dex2jarCmd.main(Dex2jarCmd.java:33)
ysh329 commented 4 years ago
$ ../dex-tools-2.1-SNAPSHOT/d2j-dex2jar.sh ./classes2.dex ./classes3.dex ./classes4.dex ./classes5.dex ./classes6.dex ./classes7.dex
dex2jar ./classes2.dex -> ./classes2-dex2jar.jar
dex2jar ./classes3.dex -> ./classes3-dex2jar.jar
dex2jar ./classes4.dex -> ./classes4-dex2jar.jar
dex2jar ./classes5.dex -> ./classes5-dex2jar.jar
dex2jar ./classes6.dex -> ./classes6-dex2jar.jar
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
dex2jar ./classes7.dex -> ./classes7-dex2jar.jar
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
WARN: ignored invalid inner class name , treat as anonymous inner class.
ysh329 commented 4 years ago

@Metadata(bv = {1, 0, 3}, d1 = {"\000&\n\002\030\002\n\002\020\000\n\002\b\002\n\002\020 \n\002\030\002\n\000\n\002\020\016\n\002\b\003\n\002\020\002\n\002\b\003\b\007\030\0002\0020\001:\001\rB\005\006\002\020\002J\016\020\b\032\n\022\004\022\0020\005\030\0010\004J\b\020\t\032\004\030\0010\007J\024\020\n\032\0020\0132\f\020\003\032\b\022\004\022\0020\0050\004J\016\020\f\032\0020\0132\006\020\006\032\0020\007R\026\020\003\032\n\022\004\022\0020\005\030\0010\004X\016\006\002\n\000R\020\020\006\032\004\030\0010\007X\016\006\002\n\000\006\016"}, d2 = {"Lcom/ss/android/ugc/aweme/effectplatform/BeautyExtra;", "", "()V", "items", "", "Lcom/ss/android/ugc/aweme/effectplatform/BeautyExtra$ItemsBean;", "resource", "", "getItems", "getResource", "setItems", "", "setResource", "ItemsBean", "tools.core_douyinCnRelease"}, k = 1, mv = {1, 1, 15})

ysh329 commented 4 years ago

@Metadata(bv = {1, 0, 3}, d1 = {"\000$\n\002\030\002\n\002\020\000\n\002\b\002\n\002\020\013\n\002\b\005\n\002\020\b\n\002\b\b\n\002\020\016\n\002\b\013\b\007\030\0002\0020\001B\005\006\002\020\002R\032\020\003\032\0020\004X\016\006\016\n\000\032\004\b\005\020\006\"\004\b\007\020\bR\032\020\t\032\0020\nX\016\006\016\n\000\032\004\b\013\020\f\"\004\b\r\020\016R\032\020\017\032\0020\nX\016\006\016\n\000\032\004\b\020\020\f\"\004\b\021\020\016R\034\020\022\032\004\030\0010\023X\016\006\016\n\000\032\004\b\024\020\025\"\004\b\026\020\027R\034\020\030\032\004\030\0010\023X\016\006\016\n\000\032\004\b\031\020\025\"\004\b\032\020\027R\032\020\033\032\0020\nX\016\006\016\n\000\032\004\b\034\020\f\"\004\b\035\020\016\006\036"}, d2 = {"Lcom/ss/android/ugc/aweme/effectplatform/BeautyExtra$ItemsBean;", "", "()V", "doubleDirectionin", "", "getDoubleDirectionin", "()Z", "setDoubleDirectionin", "(Z)V", "max", "", "getMax", "()I", "setMax", "(I)V", "min", "getMin", "setMin", "name", "", "getName", "()Ljava/lang/String;", "setName", "(Ljava/lang/String;)V", "tag", "getTag", "setTag", "value", "getValue", "setValue", "tools.core_douyinCnRelease"}, k = 1, mv = {1, 1, 15})

ysh329 commented 4 years ago
Processing retrofit2.http.Part
Processing retrofit2.http.PartMap
Processing retrofit2.http.Path
Processing retrofit2.http.Query
Processing retrofit2.http.QueryMap
Processing retrofit2.http.QueryName
Processing retrofit2.http.Streaming
Processing retrofit2.http.Url
Processing tv.danmaku.ijk.media.player.annotations.AccessedByNative
Processing tv.danmaku.ijk.media.player.annotations.CalledByNative
-----
Finished extract 7 jar file(s) to destination directory. Start Time:  Thu Dec 12 23:39:53 CST 2019 . End Time:  Thu Dec 12 23:52:52 CST 2019
ysh329 commented 4 years ago

dex4->effectplaform->BeautyExtra.class

ysh329 commented 4 years ago
# libsrlib.so:调用了1.net、2.net、3.net这3个模型
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn "1.net"
Binary file libsrlib.so matches
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn "2.net"
Binary file libsrlib.so matches
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn "3.net"
Binary file libsrlib.so matches

# 包含beauty关键词的包:libeffect.so、libttvideorecorder.so、libttvebase.so、libsscronet.so、libttvideoeditor.so、libttvideoeditor.so;
# 1. 以最小的libttvebase.so为例,其中包含beauty的方法为:_ZN22TEStickerEffectWrapper28initFaceBeautyDetectExtParamERK35bef_face_beauty_detect_ext_param_st,贴图特效(StickerEffect)、美颜(FaceBeauty)、ExtParam/face_beauty_detect_ext_param(可能是大模型或者基于更多关键点的模型的美颜)、st(单线程);
# 2. 以最大的libeffect.so为例,其中包含beauty的信息有:
bef_effect_set_beauty
bef_effect_update_beauty
bef_face_beauty_detect_ext_param *
bef_face_beauty_detect_ext_param_useV3Model_get
bef_face_beauty_detect_ext_param_useV3Model_set
beauty.json
bef-filter-name-b12-face-beauty
beautyType
beauty.json
facepartbeauty
beauty_Normal
beauty_Nature
beauty_Qingyan_State1
beauty_Qingyan_state2
beauty_Qingyan_state3
beauty_Qingyan_state4
beauty_Qingyan_state5
beauty_Qingyan_stateFinal
beauty_Qingya_Live_State1_Box1Live
beauty_QingyanLive_state2_var_live
beauty_B612_State1_scale
beauty_B612_state2_skinDetect
beauty_B612_state3_bilateralFilter
beauty_id
beauty_score
facepartbeauty
_p_bef_face_beauty_detect_ext_param_st
bef_face_beauty_detect_ext_param_st *|bef_face_beauty_detect_ext_param *
bef_face_beauty_detect_ext_param
bef_face_beauty_detect_ext_param_st::bef_face_beauty_detect_ext_param_st
bef_face_beauty_detect_ext_param_st::useV3Model
bef_face_beauty_detect_ext_param_st *
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeNone, reset m_protocol
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeNature, new IESFaceBeautyFilter
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeNormal, new IESFaceBeautyNormalFilter
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeQingyan, new QinyanBeautyFilter
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeQingyanLive, new QingyanLiveBeautyFilter
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeIES, new IESGuideFilter
--DebugBeauty--checkResExistOrLoaded: beautyName=BeautyTypeB612, new IESBeauty12OptimizeFilter
facepartbeautymodel/tt_facepartbeauty_v1.0.model
AIFacePartBeautyInfo beauty %d, score = %d
ttfaceattrmodel/tt_beauty_attr6_v1.0.model
effect [ %s ] is not beauty.
beauty_QingyanLive_state3_box2_live
beauty_B612_state4_bilateralFilter
beauty_B612_state5_Smooth_skin
/Users/chensheng/code/face_gate/smash/smash/proj/android/smash/app/src/main/jni/../../../../../../../../smash/modules/facepartbeauty/src/FacePartBeautySDK_API.cpp
tt_face_beauty_v4.0
face_part_beauty
tt_beauty_attr6_v1.0
N3BES8NtfEventI41algorithm_result_face_beauty_five_part_stEE
41algorithm_result_face_beauty_five_part_st
N3BEF29bef_filter_b12_face_beauty_stE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_0NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__baseIFvPN3BEF29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_1NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_2NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_3NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_4NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_5NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
NSt6__ndk110__function6__funcIZN3BEF17FaceBeautyFeature21checkResExistOrLoadedEvE3$_6NS_9allocatorIS4_EEFvPNS2_29bef_filter_b12_face_beauty_stEEEE
N3BES8NtfEventI34algorithm_result_facepartbeauty_stEE
34algorithm_result_facepartbeauty_st
N3BEF35bef_filter_facepartbeauty_config_stE
NSt6__ndk120__shared_ptr_pointerIPN5smash13beautyengine316HeadSegmentationEZNS2_14HeadSegHandler4InitEPKcE3$_0NS_9allocatorIS3_EEEE
ZN5smash13beautyengine314HeadSegHandler4InitEPKcE3$_0
NSt6__ndk120__shared_ptr_pointerIPN5smash13beautyengine316HeadSegmentationEZNS2_14HeadSegHandler12InitFromBuffEPKhjE3$_1NS_9allocatorIS3_EEEE
ZN5smash13beautyengine314HeadSegHandler12InitFromBuffEPKhjE3$_1
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a#
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn "\/Users/"
Binary file libeffect.so matches
Binary file libsrlib.so matches
Binary file libttffmpeg.so matches
Binary file libttvideorecorder.so matches

root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn beauty
Binary file libeffect.so matches
Binary file libttvideorecorder.so matches
libttvebase.so.T:918:0005d680 T _ZN22TEStickerEffectWrapper28initFaceBeautyDetectExtParamERK35bef_face_beauty_detect_ext_param_st
Binary file libsscronet.so matches
Binary file libttvideoeditor.so matches
Binary file libttvebase.so matches
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn face_attr
Binary file libeffect.so matches
Binary file libttvideorecorder.so matches
libttvebase.so.T:853:0005d206 T _ZN22TEStickerEffectWrapper20getSmartBeautyResultEP24bef_face_attrs_result_st
Binary file libttvebase.so matches
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn face_attr
Binary file libeffect.so matches
Binary file libttvideorecorder.so matches
libttvebase.so.T:853:0005d206 T _ZN22TEStickerEffectWrapper20getSmartBeautyResultEP24bef_face_attrs_result_st
Binary file libttvebase.so matches
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn face_detect
Binary file libeffect.so matches
Binary file libttvideorecorder.so matches
libttvebase.so.T:877:0005d5b4 T _ZN22TEStickerEffectWrapper22initFaceDetectExtParamERK28bef_face_detect_ext_param_st
Binary file libttvideoeditor.so matches
Binary file libttvebase.so matches
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn Winograd
libsrlib.so.nm:52:0003a660 T _ZN3MNN4Math17WinogradGenerater15transformWeightEPKNS_6TensorES4_
libsrlib.so.nm:53:0003a55c T _ZN3MNN4Math17WinogradGenerater20allocTransformWeightEPKNS_6TensorEiib
libsrlib.so.nm:54:00039ff8 T _ZN3MNN4Math17WinogradGeneraterC1Eiif
libsrlib.so.nm:55:00039ff8 T _ZN3MNN4Math17WinogradGeneraterC2Eiif
Binary file libsrlib.so matches
libsrlib.so.strings:189:_ZN3MNN4Math17WinogradGenerater15transformWeightEPKNS_6TensorES4_
libsrlib.so.strings:190:_ZN3MNN4Math17WinogradGenerater20allocTransformWeightEPKNS_6TensorEiib
libsrlib.so.strings:191:_ZN3MNN4Math17WinogradGeneraterC1Eiif
libsrlib.so.strings:201:_ZN3MNN4Math17WinogradGeneraterC2Eiif
libsrlib.so.strings:3289:/Users/admin1/project/inference/shrinkMNN/mnn/source/backend/cpu/compute/ConvolutionWinograd.cpp
libsrlib.so.strings:3293:/Users/admin1/project/inference/shrinkMNN/mnn/source/backend/cpu/compute/WinogradOptFunction.cpp
root@fa2dc06fa97b:~/Downloads/apk模型/douyin/aweme_aweGW_v8.8.0_55fb587/lib/armeabi-v7a# grep -rn conv3x3
libIESNN_OCL.so.strings:220:_Z9conv3x3s1P11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
libIESNN_OCL.so.nm.-D:66:000185d0 T _Z9conv3x3s1P11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
Binary file libIESNN_OCL.so matches
ysh329 commented 4 years ago
grep -rn face .
Binary file ./ttfacemodel/tt_face_v7.0.model matches
 yuanshuai06  …  aweme_aweGW_v8.8.0_55fb587  assets  model  vi ttfacemodel/tt_face_v7.0.model
 yuanshuai06  …  aweme_aweGW_v8.8.0_55fb587  assets  model  grep -rn beauty .
Binary file ./ttfaceattrmodel/tt_beauty_attr6_v1.0.model matches
 yuanshuai06  …  aweme_aweGW_v8.8.0_55fb587  assets  model  grep -rn attr .
Binary file ./ttfaceattrmodel/tt_beauty_attr6_v1.0.model matches
 yuanshuai06  …  aweme_aweGW_v8.8.0_55fb587  assets  model  grep -rn tt .
Binary file ./ttfacemodel/tt_face_extra_v10.0.model matches
Binary file ./ttfacemodel/tt_face_v7.0.model matches
Binary file ./colorcard/lookup.png matches
Binary file ./ttfaceattrmodel/tt_face_attribute_v5.0.model matches
Binary file ./ttfaceattrmodel/tt_beauty_attr6_v1.0.model matches
ysh329 commented 4 years ago
130|OnePlus7:/data/local/tmp/mnn $ /data/local/tmp/mnn/benchmark.out /data/local/tmp/mnn/model/ 100 0
MNN benchmark
Forward type: **CPU** thread=4** precision=2
--------> Benchmarking... loop = 100
[ - ] arch2.net                   max =    6.419ms  min =    4.336ms  avg =    4.525ms
[ - ] arch1.net                   max =   14.720ms  min =   13.718ms  avg =   14.089ms
Error for /home/yuanshuai/code/mnn/tools/cpp/revertMNNModel.cpp, 32
Model has no oplist
Segmentation fault

139|OnePlus7:/data/local/tmp/mnn $ md5sum ./model/*
541cb79a029bcea30a378657a23fc1ef  ./model/arch1.net
96c26aab7d5b7aa00e4e69b5d0e315e1  ./model/arch2.net
e148fe86a97f7e8a48ab77c85e4a9726  ./model/arch3.net

OnePlus7:/data/local/tmp/mnn/model $ du -sh ./*
20K ./arch1.net
20K ./arch2.net
16K ./arch3.net

根据报错信息Model has no oplist,发现Segmentation fault的位置在source/core/Interpreter.cpp:82

Interpreter* Interpreter::createFromBufferInternal(Content* net) {
    if (nullptr == net) {
        MNN_PRINT("Buffer is null for create interpreter\n");
        return nullptr;
    }
    flatbuffers::Verifier verify((const uint8_t*)(net->buffer.get()), net->buffer.size());
    if (false == VerifyNetBuffer(verify)) {
        MNN_PRINT("Invalidate buffer to create interpreter\n");
        delete net;
        return nullptr;
    }
    net->net = GetNet(net->buffer.get());
    if (nullptr == net->net->oplists()) {
        MNN_ERROR("Model has no oplist\n");
        delete net;
        return nullptr;
    }
    return new Interpreter(net);
}
ysh329 commented 4 years ago

延伸阅读


ysh329 commented 4 years ago
$ strings ./libIESNN_OCL.so | grep OCLP13sIESGPUTensor
_Z16convLayerForwardP11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z17inputLayerForwardP11sIESNet_OCLP13sIESGPUTensorP10sInference
_Z18deconvLayerForwardP11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z19eltwiseLayerForwardP11sIESNet_OCLP13sIESGPUTensorP13sEltwiseLayer
_Z21resizeBilinearForwardP11sIESNet_OCLP13sIESGPUTensorP20sResizeBilinearLayer
_Z6addPadP11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z6addPadP11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z11cutBoundaryP11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z9conv3x3s1P11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z9conv1x1s1P11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z11eltwise_sumP11sIESNet_OCLP13sIESGPUTensorP13sEltwiseLayer
_Z11deconv4x4s2P11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z14resizeBilinearP11sIESNet_OCLP13sIESGPUTensorP20sResizeBilinearLayer

$ strings ./libIESNN_OCL.so | grep Layer
_Z14parseConvLayerP7__sFILEP10sConvLayerP7sIESNet
_Z14parseReluLayerP7__sFILEP10sReluLayerP7sIESNet
_Z15parseInputLayerP7__sFILEP11sInputLayerP7sIESNet
_Z15parseScaleLayerP7__sFILEP11sScaleLayerP7sIESNet
_Z15parseSplitLayerP7__sFILEP11sSplitLayerP7sIESNet
_Z16parseConcatLayerP7__sFILEP12sConcatLayerP7sIESNet
_Z16parseDeConvLayerP7__sFILEP12sDeconvLayerP7sIESNet
_Z17parseEltwiseLayerP7__sFILEP13sEltwiseLayerP7sIESNet
_Z17parseReshapeLayerP7__sFILEP13sReshapeLayerP7sIESNet
_Z17parseSigmoidLayerP7__sFILEP13sSigmoidLayerP7sIESNet
_Z19parseBatchNormLayerP7__sFILEP15sBatchNormLayerP7sIESNet
_Z21parseDepth2SpaceLayerP7__sFILEP18sDepthtoSpaceLayerP7sIESNet
_Z21parseSpace2DepthLayerP7__sFILEP18sSpacetoDepthLayerP7sIESNet
_Z24parseResizeBilinearLayerP7__sFILEP20sResizeBilinearLayerP7sIESNet
_ZNSt6__ndk114__split_bufferI15sIESLayerTensorRNS_9allocatorIS1_EEEC2EjjS4_
_ZNSt6__ndk16vectorI15sIESLayerTensorNS_9allocatorIS1_EEE21__push_back_slow_pathIRKS1_EEvOT_
_ZNSt6__ndk16vectorI15sIESLayerTensorNS_9allocatorIS1_EEE8allocateEj
_ZNSt6__ndk16vectorI15sIESLayerTensorNS_9allocatorIS1_EEEC2ERKS4_
_Z13setFeatureMapPbPiPNSt6__ndk16vectorI15sFeaturePondMemNS1_9allocatorIS3_EEEE15sIESLayerTensor
_Z16convLayerForwardP11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z17inputLayerForwardP11sIESNet_OCLP13sIESGPUTensorP10sInference
_Z18deconvLayerForwardP11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z19eltwiseLayerForwardP11sIESNet_OCLP13sIESGPUTensorP13sEltwiseLayer
_Z21resizeBilinearForwardP11sIESNet_OCLP13sIESGPUTensorP20sResizeBilinearLayer
_Z6addPadP11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z6addPadP11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z11cutBoundaryP11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z9conv3x3s1P11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z9conv1x1s1P11sIESNet_OCLP13sIESGPUTensorP10sConvLayer
_Z11eltwise_sumP11sIESNet_OCLP13sIESGPUTensorP13sEltwiseLayer
_Z11deconv4x4s2P11sIESNet_OCLP13sIESGPUTensorP12sDeconvLayer
_Z14resizeBilinearP11sIESNet_OCLP13sIESGPUTensorP20sResizeBilinearLayer
ysh329 commented 4 years ago

直接在lib/armeabi-v7a目录下使用grep命令检索关键词:Conv、Convolution、face_detect、Beauty、beauty、inference、predict等,最后定位到以下3个so:

注:根据目录名片段,可检索到一大把姓名拼音缩写,以及对应模块代码,若运气好在微信里以拼音可检索到对应人名(含未加好友的群成员名的拼音),核对信息大概率是这个人,或许我可以挖人了。

libsrlib

对libsrlib.so使用vim命令直接打开观察,再配合stings、nm、grep等命令组合,检索到有用到MNN引擎,以及shrinkMNN关键词,猜测对MNN压缩删除未使用到的代码。

并在其中找到几个文件名:1.net、2.net、3.net,对应assets/目录下的三个文件(arch1.net、arch2.net、arch3.net),更多见后文。

libIESNN_OCL

对libIESNN_OCL.so使用stings、nm、grep等命令组合,可以确认推理引擎名为IESNN,CPU-armv7a支持14个CPU层和4个GPU(OpenCL)层,因为包含Input层,而且函数名带有Layer,估计最早支持的应该也是Caffe。

ysh329 commented 4 years ago
ysh329 commented 4 years ago

Android aapt - eLinux.org https://elinux.org/Android_aapt