search

mamoe / mirai

高效率 QQ 机器人支持库
https://mirai.mamoe.net
GNU Affero General Public License v3.0
14.42k stars 2.54k forks source link

ProGuard 去除了 `MiraiImpl` 的 kotlin 信息导致 `IMirai` 无法初始化 #1861

Closed zhaodice closed 2 years ago

zhaodice commented 2 years ago

我是在安卓平台上编译运行的mirai 非常诡异的错误，在Mirai2.9.0-RC2上是正常的，但更新到Mirai2.10.0-RC2的时候，在app端调用BotFactory 就会出现这种错误，但更诡异的是，在插件内调用BotFactory进行登录则一切正常，问题可能出在哪里呢？ 错误信息：java.lang.NullPointerException: Attempt to invoke interface method 'net.mamoe.mirai.BotFactory net.mamoe.mirai.IMirai.getBotFactory()' on a null object reference

zhaodice commented 2 years ago

APP端的登录代码

runBlocking(MiraiConsole.coroutineContext) {
  loginBot?.closeAndJoin()
  loginBot = BotFactory.newBot(qq.toLong(), password) {
...

插件端的登录代码：

....
BotFactory.newBot
....
cssxsh commented 2 years ago

看起来像是缺依赖导致 IMirai 没加载上

zhaodice commented 2 years ago

看起来像是缺依赖导致 IMirai 没加载上

如果是缺依赖，为什么在插件内调用登录却一切正常呢？

zhaodice commented 2 years ago

问题就诡异在，插件内调用登录一切正常，插件外调用（也就是mirai本体环境）却空指针错误

cssxsh commented 2 years ago

问题就诡异在，插件内调用登录一切正常，插件外调用（也就是mirai本体环境）却空指针错误

你在安卓APP 里内嵌了一个 Mirai-Console ？

zhaodice commented 2 years ago

问题就诡异在，插件内调用登录一切正常，插件外调用（也就是mirai本体环境）却空指针错误

你在安卓APP 里内嵌了一个 Mirai-Console ？

没错，我在console之外(mirai环境)调用登录就出错，但在console内(插件环境内，也就是onEnable那个地方)调用登录就正常

zhaodice commented 2 years ago

其实APP的整体结构和MiraiAndroid差不多

zhaodice commented 2 years ago

这个问题是在2.10-RC的时候就开始出现的，2.9.0-RC2以及之前的版本一切正常

Him188 commented 2 years ago

2.10 应该没有改这个加载策略

zhaodice commented 2 years ago

顺带一提，我的console版本依然是2.6.0，但我觉得关系应该不是很大,因为console原则上不应该影响core的api（况且在console插件内调用登录还是正常的）

Karlatemp commented 2 years ago

有无稍微详细点的错误堆栈

Karlatemp commented 2 years ago

你使用的 kotlin-gradle 版本为?

Him188 commented 2 years ago

提供 build.gradle

zhaodice commented 2 years ago

有无稍微详细点的错误堆栈

堆栈，这个...我再看一下，当时没保存...，但我确定是在调用 BotFactory.newBot 的时候出的问题（你们顺着这个入口进去追溯一下应该知道在怎么地方调用了IMirai？） kotlin的依赖我引入了这些包

2022/01/29  22:27         3,038,560 kotlin-reflect-1.6.10.jar
2022/01/29  22:28         1,508,076 kotlin-stdlib-1.6.10.jar
2022/01/29  22:28         1,488,107 kotlinx-coroutines-core-jvm-1.6.0.jar
2021/11/22  19:12            17,709 kotlinx-coroutines-io-0.1.16.jar
2021/11/22  19:12           421,729 kotlinx-coroutines-io-jvm-0.1.16.jar
2022/01/29  22:29           342,172 kotlinx-serialization-core-jvm-1.3.2.jar
2022/01/29  22:29           212,236 kotlinx-serialization-json-jvm-1.3.2.jar
2022/01/29  22:30            94,493 kotlinx-serialization-protobuf-jvm-1.3.2.jar

然后是build.gradle(其实没有意义，因为我直接使用kotlin库的jar文件）

buildscript {
    ext {
        //Mirai 2.7+

        kotlin_version = '1.6.0'
        kotlin_coroutines_version = '1.6.0'
        kotlinx_serialization_version = '1.2.1'
        androidx_lifecycle_version = '2.3.1'
        androidx_navigation_version = '2.3.5'
    }
...
Him188 commented 2 years ago

提供完整的 build.gradle，你觉得没用的信息可能我觉得有用

zhaodice commented 2 years ago

这是app下的build.gradle

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'

android {
    compileSdkVersion 30
    compileOptions {
        coreLibraryDesugaringEnabled true
        sourceCompatibility = 1.8
        targetCompatibility = 1.8
    }
    kotlinOptions {
        jvmTarget = "1.8"
        freeCompilerArgs = ['-Xjvm-default=all','-Xopt-in=kotlin.RequiresOptIn']
    }
    defaultConfig {
        multiDexEnabled true
        minSdkVersion 21
        versionCode 10
        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
        flavorDimensions "versionCode"
        externalNativeBuild {
            cmake {
                //cppFlags "-std=c++11 -fexceptions -pthread"
                /**
                 * -DHAVE_CONFIG_H这个有个坑 lame配置这个参数编译不通过 libmad不加这个参数转码会失真
                 */
                cFlags "-DSTDC_HEADERS -DHAVE_CONFIG_H -DFPM_DEFAULT"
            }
            ndk {
            }
        }
    }
    buildTypes {
        release {
            zipAlignEnabled true
            minifyEnabled true
            shrinkResources true
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }
    packagingOptions {
        exclude "META-INF/**"
        exclude "WEB-INF/**"
        exclude "okhttp3/**"
        exclude "net/**"
        exclude "org/**"
        exclude 'DebugProbesKt.bin'
        merge 'org/mozilla/javascript/resources/Messages.properties'
    }

    productFlavors {
        miraiCore {
            applicationId "org.mirai.zhao.dice"
            versionName "4.3.7"
            targetSdkVersion 29//改成30将无法访问/sdcard根目录！
            resValue "string", "app_name", "DICE X4"
            manifestPlaceholders = [
                    FRAME_TYPE : "MiraiCore"
            ]
            ndk {
                abiFilters "armeabi-v7a", "arm64-v8a"
            }
        }
        miraiGo {
            applicationId "org.mirai.zhao.gdice"
            versionName "4.0.4"
            targetSdkVersion 29
            resValue "string", "app_name", "GDICE X4"
            manifestPlaceholders = [
                    FRAME_TYPE : "MiraiGo"
            ]
            ndk {
                abiFilters "arm64-v8a","armeabi-v7a"
            }
        }
    }
    applicationVariants.all { variant ->
        variant.outputs.all { output ->
            def outputFile = output.outputFile
            def fileName = "Dice_unknown"
            def appName = "unknown"
            if (outputFile != null && outputFile.name.endsWith('.apk')) {
                if (productFlavors.applicationId[0] == 'org.mirai.zhao.gdice') {
                    appName = "GDice"
                }else if (productFlavors.applicationId[0] == 'org.mirai.zhao.dice') {
                    appName = "Dice"
                }
                if (variant.buildType.name == 'release') {//如果是release包
                    fileName = "${appName}_release_v${productFlavors.versionName[0]}.apk"
                } else if (variant.buildType.name == 'debug') {//如果是debug包
                    fileName = "${appName}_debug_v${productFlavors.versionName[0]}.apk"
                }
                // 自定义输出路径
                //variant.getPackageApplicationProvider().get().outputDirectory = new File(project.rootDir.absolutePath + File.separator + "/build/release")
                outputFileName = fileName
            }
        }
    }
}
repositories {
    maven { url 'https://jitpack.io' }
}
//
configurations {
    cleanedAnnotations
    implementation.exclude group: 'org.jetbrains', module:'annotations'
    implementation.exclude group: 'org.jetbrains.kotlinx', module:'kotlinx-coroutines-core-jvm'
    implementation.exclude group: 'org.jetbrains.kotlin', module:'kotlin-stdlib'
}
kotlin.sourceSets.all {
    languageSettings.useExperimentalAnnotation("net.mamoe.mirai.console.ConsoleFrontEndImplementation")
    languageSettings.useExperimentalAnnotation("net.mamoe.mirai.console.util.ConsoleInternalApi")
    languageSettings.useExperimentalAnnotation("net.mamoe.mirai.console.util.ConsoleExperimentalApi")
    languageSettings.useExperimentalAnnotation("net.mamoe.mirai.utils.MiraiInternalApi")
    languageSettings.useExperimentalAnnotation("net.mamoe.mirai.utils.MiraiExperimentalApi")
}
dependencies {
    implementation (fileTree(dir: "libs", include: ["*.jar"]))
    implementation (fileTree(dir: "../allLibs/ext", include: ["ext-gson-*.jar",
                                                              "ext-luaj-*.jar",
                                                              "ext-snakeyaml-*.jar",
                                                              "ext-toml4j-*.jar"]))
    implementation (fileTree(dir: "../allLibs/kotlin", include: ["*.jar"]))
    try{
        miraiCoreImplementation (fileTree(dir: "../allLibs/miraiConsole", include: ["*.jar"]))
        miraiCoreImplementation (fileTree(dir: "../allLibs/miraiLibs", include: ["*.jar"]))
    }catch(Throwable ignored){

    }
    try{
        miraiGoImplementation "com.koushikdutta.async:androidasync:3.1.0"
        //miraiGoImplementation project(":miraigo-zhao") //动态dex加载
    }catch(Throwable ignored){

    }
    //implementation project(':lib_permisshelper')

    implementation 'org.greenrobot:eventbus:3.0.0'
    implementation 'com.google.android.material:material:1.0.0'
    implementation 'androidx.swiperefreshlayout:swiperefreshlayout:1.1.0'
    implementation 'io.apisense:rhino-android:1.1.1'
    implementation 'androidx.annotation:annotation:1.2.0'
    implementation 'androidx.appcompat:appcompat:1.3.1'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.0'
    implementation "androidx.lifecycle:lifecycle-livedata-ktx:$androidx_lifecycle_version"
    implementation "androidx.lifecycle:lifecycle-viewmodel-ktx:$androidx_lifecycle_version"
    implementation "androidx.navigation:navigation-fragment-ktx:$androidx_navigation_version"
    implementation "androidx.navigation:navigation-ui-ktx:$androidx_navigation_version"
    implementation 'cat.ereza:customactivityoncrash:1.3.0'
    coreLibraryDesugaring 'com.android.tools:desugar_jdk_libs:1.1.5'
}

这是主项目的gradle


// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    ext {
        //Mirai 2.7+

        kotlin_version = '1.6.0'
        kotlin_coroutines_version = '1.6.0'
        kotlinx_serialization_version = '1.2.1'
        androidx_lifecycle_version = '2.3.1'
        androidx_navigation_version = '2.3.5'
    }

    repositories {
        maven { url 'https://maven.aliyun.com/repository/google' }
        maven { url 'https://maven.aliyun.com/repository/public' }
        //mavenCentral()
        //google()
        //jcenter()
    }

    dependencies {
        classpath 'com.github.jengelman.gradle.plugins:shadow:6.1.0'
        classpath 'com.android.tools.build:gradle:7.0.3'
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.0"//不得轻易升级，影响兼容性！！
        //classpath "org.jetbrains.kotlin:kotlin-serialization:$kotlin_version"
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        maven { url 'https://maven.aliyun.com/repository/google' }
        maven { url 'https://maven.aliyun.com/repository/public' }
        maven { url 'https://maven.aliyun.com/repository/gradle-plugin' }
        //google()
        //jcenter()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}
zhaodice commented 2 years ago

其中allLibs文件夹下的文件结构

─ext
│      ext-gson-2.8.8.jar
│      ext-json-20160810.jar
│      ext-luaj-3.0.1.jar
│      ext-snakeyaml-1.26.jar
│
├─kotlin
│      kotlin-reflect-1.6.10.jar
│      kotlin-stdlib-1.6.10.jar
│      kotlinx-coroutines-core-jvm-1.6.0.jar
│      kotlinx-coroutines-io-0.1.16.jar
│      kotlinx-coroutines-io-jvm-0.1.16.jar
│      kotlinx-serialization-core-jvm-1.3.2.jar
│      kotlinx-serialization-json-jvm-1.3.2.jar
│      kotlinx-serialization-protobuf-jvm-1.3.2.jar
│
├─miraiConsole
│      mirai-CheckerConstants.jar
│      mirai-console-2.6.0-dalvik.jar
│      mirai-yamlkt-jvm-0.7.5.jar
│
└─miraiLibs
        mirai-core-android-2.9.0-RC2.jar
        mirai-core-api-android-2.9.0-RC2.jar
        mirai-core-utils-android-2.9.0-RC2.jar
        mirai-dependents.jar
Him188 commented 2 years ago

不要通过 fileTree 添加 jar 依赖，这很有可能就是问题原因

zhaodice commented 2 years ago

不要通过 fileTree 添加 jar 依赖，这很有可能就是问题原因

主要是，我觉得我的开发制程...没有那么标准，为简化问题便没直接贴出来，见笑了 之前一直这样用的，都是正常的，这次就失灵了...明天我再试试看用implementation 引用包试试看吧（主要是这个强国总是干扰我同步，所以才直接用jar出此下策）

zhaodice commented 2 years ago

不要通过 fileTree 添加 jar 依赖，这很有可能就是问题原因

好像没有用，我直接 implementation("net.mamoe:mirai-core:2.10.0") 之后 一样是空指针错误，我稍后贴出详细错误堆栈

zhaodice commented 2 years ago

错误信息

java.lang.NullPointerException: Attempt to invoke interface method 'net.mamoe.mirai.BotFactory net.mamoe.mirai.lMirai.getBotFactory()' on a null object reference
at net.mamoe.mirai.BotFactory$INSTANCE.newBot()
at org.mirai.zhao.dice.console.MiraiCoreConsoleService$iConsoleServiceStub$1$loginBot$1$1.invokeSuspend()
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith()
at kotlinx.coroutines.DispatchedTask.run()
at kotlinx.coroutines.EventLooplmplBase.processNextEvent()
at kotlinx.coroutines.BlockingCoroutine.joinBlocking()
at kotlinx.coroutines.BuildersKt_ BuildersKt.runBlocking()
at kotlinx.coroutines.BuildersKt.runBlocking()
at kotlinx.coroutines. BuildersKt_ BuildersKt.runBlocking$default()
zhaodice commented 2 years ago

图片版本（走的图片识别，文字版本可能有错别字?)： QQ图片20220223151817

zhaodice commented 2 years ago

之前都是好好的，Mirai2.10.0-RC就开始不行了，不太清楚这个版本更新了什么东西导致的

zhaodice commented 2 years ago

解决了,我跟着错误信息找到了相关源码文件 https://github.com/mamoe/mirai/blob/7a32afc14c59618a726da68c957ec7d237224fde/mirai-core-api/src/commonMain/kotlin/IMirai.kt 发现了提示，我在初始化的时候调用了

@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // 必要
net.mamoe.mirai._MiraiInstance.set(net.mamoe.mirai.internal.MiraiImpl())

问题解决，虽然还是不知道这是为什么，之前版本明明不需要的，为什么2.10-RC就要这样（如果这是新版本无关紧要的问题的话就可以close了）

Karlatemp commented 2 years ago

提供能复现错误的最终 jar & apk 成品，可能是因为 proguard

zhaodice commented 2 years ago

提供能复现错误的最终 jar & apk 成品，可能是因为 proguard

但是，当时测试，我加了一句这个

-keep class ** { *; }

(我没法把混淆关掉，因为关掉了会出现其他未知错误，总之我这边编译环境乱糟糟的）

zhaodice commented 2 years ago

编译成品

https://cowtransfer.com/s/061631e1ad0942

Him188 commented 2 years ago

我看了代码，基于 Kotlin null 检查，上述 NPE 绝对不可能出现。怀疑是 ProGuard 优化字节码导致改变了行为。总之你那边构建环境太复杂了，我也不好分析。

zhaodice commented 2 years ago

我看了代码，基于 Kotlin null 检查，上述 NPE 绝对不可能出现。怀疑是 ProGuard 优化字节码导致改变了行为。总之你那边构建环境太复杂了，我也不好分析。

说到 Kotlin null 检查，可能是因为这个地方吧？总之问题解决就好了

#去掉空检查
-assumenosideeffects class kotlin.jvm.internal.Intrinsics {
    static void checkNotNull(...);
    static void checkParameterIsNotNull(...);
    static void checkNotNullExpressionValue(...);
    static void checkNotNullParameter(...);
    static void checkExpressionValueIsNotNull(...);
    static void throwUninitializedPropertyAccessException(...);
}
Him188 commented 2 years ago

题外话：为什么要去掉 null 检查？这是 fail-fast 行为而且这些检查速度很快

Karlatemp commented 2 years ago 
.class public final Lnet/mamoe/mirai/_MiraiInstance;
.super Ljava/lang/Object;
.source ""
.method public static final get()Lnet/mamoe/mirai/IMirai;
    .locals 1
    .annotation runtime Lkotlin/jvm/JvmStatic;
    .end annotation

    .annotation build Lorg/jetbrains/annotations/NotNull;
    .end annotation

    sget-object v0, Lnet/mamoe/mirai/_MiraiInstance;->instance:Lnet/mamoe/mirai/IMirai;

    if-nez v0, :cond_0

-   invoke-static {}, Lnet/mamoe/mirai/FindMiraiInstance_androidKt;->findMiraiInstance()Lnet/mamoe/mirai/IMirai;

    move-result-object v0

    sput-object v0, Lnet/mamoe/mirai/_MiraiInstance;->instance:Lnet/mamoe/mirai/IMirai;

    :cond_0
    return-object v0
.end method
.class public final Lnet/mamoe/mirai/FindMiraiInstance_androidKt;
.super Ljava/lang/Object;
.source ""

# direct methods
.method public static final synthetic findMiraiInstance()Lnet/mamoe/mirai/IMirai;
    .locals 2

    const-class v0, Lnet/mamoe/mirai/IMirai;

    invoke-static {v0}, Ljava/util/ServiceLoader;->load(Ljava/lang/Class;)Ljava/util/ServiceLoader;

    move-result-object v0

    invoke-static {v0}, Lkotlin/collections/CollectionsKt;->firstOrNull(Ljava/lang/Iterable;)Ljava/lang/Object;

    move-result-object v0

    check-cast v0, Lnet/mamoe/mirai/IMirai;

    if-nez v0, :cond_0

    const-class v0, Lnet/mamoe/mirai/internal/MiraiImpl;

    sget-object v1, Lnet/mamoe/mirai/internal/MiraiImpl;->INSTANCE:Lnet/mamoe/mirai/internal/MiraiImpl$INSTANCE;

    invoke-static {v0}, Lkotlin/jvm/JvmClassMappingKt;->getKotlinClass(Ljava/lang/Class;)Lkotlin/reflect/KClass;

    move-result-object v0

    invoke-static {v0}, Lkotlin/reflect/full/KClasses;->getCompanionObjectInstance(Lkotlin/reflect/KClass;)Ljava/lang/Object;

    move-result-object v0

    check-cast v0, Lnet/mamoe/mirai/IMirai;

    :cond_0
    return-object v0
.end method

https://github.com/mamoe/mirai/blob/96e943c33fb14283d8016339e8f17d954fc8ec1f/mirai-core-api/src/androidMain/kotlin/findMiraiInstance.android.kt#L15-L19

经过最终路由追踪确定原因为 ProGuard 祛除了 MiraiImpl 的相关 kotlin 注解, 导致 KClass<*>.companionObjectInstance 返回 null, 并且因为祛除了空检查导致没有在请求服务的时候直接报错

Him188 commented 2 years ago

这个应该算是 ProGuard 的 bug