ankidroid / Anki-Android

AnkiDroid: Anki flashcards on Android. Your secret trick to achieve superhuman information retention.
GNU General Public License v3.0
8.2k stars 2.17k forks source link

API not working - NullPointerException on addNote or addNotes #14787

Closed Linkle123 closed 8 months ago

Linkle123 commented 8 months ago

Checked for duplicates?

What are the steps to reproduce this bug?

Two parts to this bug. I think one of them is dependent on the other. Nearly all of the recent releases of the github on jitpack failed build, so I can't access newer versions of the api, which I suspect the second bug is resolved in.

I checked the jitpack at https://jitpack.io/com/github/ankidroid/Anki-Android/ to see which versions of the dependency I could access, and found that only these versions actually built and are importable: Versions that resolve v2.14.13 v2.14alpha21 2.15alpha16 api-v1.1.0 2.12alpha2 2.10alpha17 2.9alph 2.9alpha67 2.9alpha55

I haven't checked addNotes function on all of them, but on api-v1.1.0 which is what's in the API docs, and 2.15alpha16, the most recent one that succeeded in build, this bug still appeared. After digging into the stack trace and looking online, I suspect it has to do with the fact that kotlin uses kotlin.String, while java uses java.lang.String, which makes the Parcel class crash while trying to unpack the kotlin.String. This hypothesis is further supported by the fact that 2.15alpha16 is still using ApiContent.java, while the newest version uses ApiContent.kt.

I found someone else who had the same issue here: https://github.com/ankidroid/Anki-Android/issues/12756#issue-1428888216 But it was never resolved, and I suspect it's because of this: https://github.com/ankidroid/Anki-Android/issues/7929#issue-771670899 It's been 3 years since the other API not publishing issue's been posted, and this also involves the regular jitpack not publishing, so I don't think this counts as a duplicate.

To reproduce, Just send any basic request to either addNotes function, and it fails with a NullPointerException.

here's my gradle file if that helps.

plugins {
    id("com.android.application")
    id("org.jetbrains.kotlin.android")
    id("com.google.android.libraries.mapsplatform.secrets-gradle-plugin")
}

android {
    namespace = "secret"
    compileSdk = 33

    defaultConfig {
        applicationId = "secret"
        minSdk = 24
        targetSdk = 33
        versionCode = 1
        versionName = "1.0"

        testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
        vectorDrawables {
            useSupportLibrary = true
        }
    }

    buildTypes {
        release {
            isMinifyEnabled = false
            proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
            )
        }
    }
    compileOptions {
        sourceCompatibility = JavaVersion.VERSION_1_8
        targetCompatibility = JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = "1.8"
    }
    buildFeatures {
        compose = true
    }
    composeOptions {
        kotlinCompilerExtensionVersion = "1.4.3"
    }
    packaging {
        resources {
            //excludes += "/META-INF/{AL2.0,LGPL2.1}"
            excludes += "/META-INF/*"
        }
    }
}

dependencies {

    implementation("androidx.core:core-ktx:1.9.0")
    implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.6.2")
    implementation("androidx.activity:activity-compose:1.7.2")
    implementation(platform("androidx.compose:compose-bom:2023.03.00"))
    implementation("androidx.compose.ui:ui")
    implementation("androidx.compose.ui:ui-graphics")
    implementation("androidx.compose.ui:ui-tooling-preview")
    implementation("androidx.compose.material3:material3")
    implementation("androidx.constraintlayout:constraintlayout:2.1.4")
    implementation("androidx.recyclerview:recyclerview:1.3.2")

    testImplementation("junit:junit:4.13.2")
    androidTestImplementation("androidx.test.ext:junit:1.1.5")
    androidTestImplementation("androidx.test.espresso:espresso-core:3.5.1")
    androidTestImplementation(platform("androidx.compose:compose-bom:2023.03.00"))
    androidTestImplementation("androidx.compose.ui:ui-test-junit4")
    debugImplementation("androidx.compose.ui:ui-tooling")
    debugImplementation("androidx.compose.ui:ui-test-manifest")

    // Ankidroid
    implementation("com.github.ankidroid:Anki-Android:2.15alpha16")
    // Versions that resolve v2.14.13 v2.14alpha21 2.15alpha16 api-v1.1.0 2.12alpha2 2.10alpha17 2.9alph 2.9alpha67 2.9alpha55
    // Google translate
    implementation("com.google.cloud:google-cloud-translate:2.30.0")

}

Expected behaviour

Adds note to a deck in Ankidroid

Actual behaviour

Crash with NullpointerException stack trace:

java.lang.RuntimeException: java.lang.NullPointerException
                                                                                                        at secret.MainActivity.saveAllCards$lambda$7(MainActivity.kt:182)
                                                                                                        at secret.MainActivity.$r8$lambda$GmebkLHNZfWs6VtXRwygRRvhs00(Unknown Source:0)
                                                                                                        at secret.MainActivity$$ExternalSyntheticLambda4.run(Unknown Source:6)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                                                                        at java.lang.Thread.run(Thread.java:1012)
                                                                                                    Caused by: java.lang.NullPointerException
                                                                                                        at android.os.Parcel.createExceptionOrNull(Parcel.java:3029)
                                                                                                        at android.os.Parcel.createException(Parcel.java:3007)
                                                                                                        at android.os.Parcel.readException(Parcel.java:2990)
                                                                                                        at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:190)
                                                                                                        at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:142)
                                                                                                        at android.content.ContentProviderProxy.insert(ContentProviderNative.java:557)
                                                                                                        at android.content.ContentResolver.insert(ContentResolver.java:2200)
                                                                                                        at android.content.ContentResolver.insert(ContentResolver.java:2162)
                                                                                                        at com.ichi2.anki.api.AddContentApi.addNoteForContentValues(AddContentApi.java:110)
                                                                                                        at com.ichi2.anki.api.AddContentApi.addNoteInternal(AddContentApi.java:106)
                                                                                                        at com.ichi2.anki.api.AddContentApi.addNote(AddContentApi.java:92)

Debug info

AnkiDroid Version = 2.16.5 (953b9bd879269910ee962b520da9705336d8bc2d)
Android Version = 13
ProductFlavor = play
Manufacturer = samsung
Model = SM-S916U1
Hardware = qcom
Webview User Agent = Mozilla/5.0 (Linux; Android 13; SM-S916U1 Build/TP1A.220624.014; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/119.0.6045.66 Mobile Safari/537.36
ACRA UUID = 7936b50d-7236-4f41-a77a-0731e8028149
New schema = false
Scheduler = std2
Crash Reports Enabled = true
DatabaseV2 Enabled = true

(Optional) Anything else you want to share?

No response

Research

welcome[bot] commented 8 months ago

Hello! πŸ‘‹ Thanks for logging this issue. Please remember we are all volunteers here, so some patience may be required before we can get to the issue. Also remember that the fastest way to get resolution on an issue is to propose a change directly, https://github.com/ankidroid/Anki-Android/wiki/Contributing

mikehardy commented 8 months ago

Jitpack is attempting to build with an unsupported JDK, it defaults to 11 and we require 17 now

From the most recent build log - 2.17alpha3 right now:


ERROR: AnkiDroid builds with JVM version 17 or 18.
  Incompatible major version detected: '11'

  If you receive this error because you want to use a newer JDK, we may accept PRs to support new versions.
  Edit the main build.gradle file, find this message in the file, and add support for the new version.
  Please make sure the `jacocoTestReport` target works on an emulator with our minSdkVersion (currently 21).

We can set what java we need with a jitpack.yml file: https://docs.jitpack.io/building/#java-version

That should clear up the inability to build.

Our documentation includes instructions for the tag api-v1.1.0 which according to the jitpack.io build log was build in 2020 πŸ˜†

Start: Sun Dec 20 19:15:48 UTC 2020 74ecdc37378c

...so I suspect that won't work.

I must admit I'm not too hopeful that current versions work as there are a couple bugs open on the API but sadly I haven't had time to work through API usage and neither has anyone else for a while

mikehardy commented 8 months ago

I've got jitpack using correct JDK now but their build command fails. They use this (when trying to compile against my branch where I'm looking at this):

./gradlew clean -Pgroup=com.github.mikehardy -Pversion=cd3bd5e36f -xtest -xlint assemble publishToMavenLocal

Had to rewrite everything but I've got it now. PR landing shortly, 2.17alpha6 should have a correctly generated api aar release artifact on jitpack https://jitpack.io/com/github/mikehardy/Anki-Android/pin-jitpack-build-jdk-v2.5.4-gcd17835-10087/build.log

mikehardy commented 8 months ago

At some point in the next hour or so you should be able to hit this URL: https://jitpack.io/com/github/ankidroid/Anki-Android/v2.17alpha6 as a dependency

The first time you try it won't be built, but a few minutes after you try it, the artifact should be available with build.log here https://jitpack.io/com/github/ankidroid/Anki-Android/v2.17alpha6/build.log

[edit: it is up and built successfully...]

I'll be curious to hear what fails as you move forward onto next steps trying to use it 😬

Linkle123 commented 8 months ago

I can now import v2.17alpha6 as a dependency, but it still fails to addNotes. Same as before, NullPointerException. Is there any way to change it to not use Parcels? I think that's where the issue lies

Linkle123 commented 8 months ago

I think I may have found the issue. In ContentProviderNative.java, in this function: public int bulkInsert(@NonNull AttributionSource attributionSource, Uri url, ContentValues[] values) throws RemoteException { at this line url.writeToParcel(data, 0);, if you follow the stack trace to Uri.java, you can see that at public void writeToParcel(Parcel parcel, int flags) { on line fragment.writeTo(parcel);, the encoded and decoded values in the writeTo method evaluate to null, but instead of throwing an exception at `throw new IllegalArgumentException("Unknown representation: "

mikehardy commented 8 months ago

You may have better luck with 2.17alpha8 (which is not published yet, but will be as soon as this PR merges) - #14836

I'm not 100% that was the cause of your error, but the API was definitely not working without it

Linkle123 commented 8 months ago

Still getting the same error after I downloaded the APK. Maybe try also uploading to jitpack? Maybe it won't work unless the dependency also has the fix

mikehardy commented 8 months ago

Still getting the same error after I downloaded the APK. Maybe try also uploading to jitpack? Maybe it won't work unless the dependency also has the fix

I don't believe that's how jitpack works (we don't upload there, they dynamically attempt to build + serve whatever github tag you request, on their side) and I don't believe that's how the API works, the API artifact you pull from jitpack is mostly just a contract, the implementation change here "lives" (executes) inside the AnkiDroid app that is installed as the package named "com.ichi2.anki.AnkiDroid" on the system

So if you are pulling jitpack artifact https://jitpack.io/com/github/ankidroid/Anki-Android/v2.17alpha8 (which I don't think you are as I just hit that URL and it built it for the first time indicating it is the first request ever for the artifact) and you have 2.17alpha8 of AnkiDroid installed as the main install (not a parallel build or similar...) then at least you are using the up to date code everywhere

I think what is happening in this issue vs the other issues just fixed is that the content you are sending is missing some contractually oblligated items, as indicated by the nulls you saw when diving in to the content provider code. The error handling perhaps needs some help so that is more obvious but I wonder if you looked at how the sample app inserts cards and just tried that first ("simply problem" + "copy should-be-working solution" troubleshooting strategy) - maybe that would work

Linkle123 commented 8 months ago

I included the new dependency now anyways just in case, but doesn't seem to have changed anything.

I don't think I'm missing any items. I'm using the Basic model which only needs a front and a back, which I confirmed by printing out the getFieldsList(modelId) to Log. I also back-checked whether the model names and deck names for the ids I got were correct by using api.getModelName and api.getDeckName, and they matched the strings I put in.

image

Then, in the debugger, the values I sent in to addNotes all look fine. Both the deckId and modelId are there, the fieldslist I sent is there.

image

I also confirmed it was compacted into the newNotesValuesList correctly with a US

image

I thought maybe some kind of bug was making tag non-nullable, so I did the same thing and added tags too, but it still had the same error.

I don't see how I could be missing any contractually obligated items. I also initialized the api in Oncreate with ankiAPI = AddContentApi(this) I don't think it should be an issue that I initialized it in Oncreate since all of the other api calls work.

Am I missing something super obvious here or something? The only possible thing I can think of at this point is that Parcel can't read the universal separator (US) for some reason, but I highly doubt that's the case. I'm so lost.

Linkle123 commented 8 months ago

Here's all the relevant code `fun saveAllCards() { val executor = Executors.newSingleThreadExecutor() val handler = Handler(Looper.getMainLooper())

    executor.execute {
        val deckId: Long? = getOrGenerateDeckId()
        val modelId: Long? = getOrGenerateModelId()
        var failedApi = false

        if ((deckId == null) || (modelId == null)) {
            failedApi = true
            executor.shutdown()
        }

        val x = ankiAPI.getFieldList(modelId!!)
        for(y in x!!){
            Log.i("field_type", y)
        }
        val a = ankiAPI.getModelName(modelId)
        Log.i("model_name", a!!)
        val b = ankiAPI.getDeckName(deckId!!)
        Log.i("deck_name", b!!)
        var convertedCards = convertFlashcardsToStringArray()
        val filteredCards = handleDuplicateCards(convertedCards, modelId!!).toList()
        if (filteredCards.size != 0) {
            //val tag = mutableListOf<Set<String>>()
            for (card in filteredCards) {
                //tag.add(listOf("Translation_to_Flashcard_App").toSet())
                for (field in card)
                    Log.i("field", field)
            }
            Log.i("deckId", deckId.toString())
            Log.i("modelid", modelId.toString())
            //val converterForApi = KotlintoJavaApiConverter(ankiAPI)
            //converterForApi.addNotesWithJavaObjects(deckId!!, modelId, filteredCards)
            ankiAPI.addNotes(deckId!!, modelId!!, filteredCards, null)
        }
        handler.post{
            if(failedApi){
                Toast.makeText(
                    this,
                    getResources().getString(R.string.card_add_fail),
                    Toast.LENGTH_LONG
                ).show()
            }
            else {
                clearArray()
            }
        }
    }
}

fun getOrGenerateDeckId(): Long? {
    var deckId: Long? = getDeckId(this.resources.getString(R.string.deck_name))
    if (deckId == null) {
        deckId = ankiAPI.addNewDeck(this.resources.getString(R.string.deck_name))
    }
    return deckId
}

fun getOrGenerateModelId(): Long?{
    var modelId: Long? = getModelId(this.resources.getString(R.string.model_name))
    if (modelId == null) {
        modelId = ankiAPI.addNewBasicModel(this.resources.getString(R.string.model_name))
    }
    return modelId
}

fun getModelId(modelName: String): Long? {
    val modelList: Map<Long, String>? = ankiAPI.modelList
    if (modelList != null) {
        for ((key, value) in modelList) {
            if (value.equals(modelName, ignoreCase = true)) {
                return key
            }
        }
    }
    return null
}

fun getDeckId(deckName: String): Long? {
    val deckList: Map<Long, String>? = ankiAPI.deckList
    if (deckList != null) {
        for ((key, value) in deckList) {
            if (value.equals(deckName, ignoreCase = true)) {
                return key
            }
        }
    }
    return null
}`
david-allison commented 8 months ago
            //converterForApi.addNotesWithJavaObjects(deckId!!, modelId, filteredCards)
            ankiAPI.addNotes(deckId!!, modelId!!, filteredCards, null)

The signature is ankiAPI.addNotes(modelId!!, deckId!!, filteredCards, null)

https://github.com/ankidroid/Anki-Android/blob/a61510d9ca120d82154fc3f0e35d7a227879dede/api/src/main/java/com/ichi2/anki/api/AddContentApi.kt#L96-L129

Fixed in #14847

mikehardy commented 8 months ago

So it looks like with 2.17alpha9 we'll have a better error message here however based on the comments above I think if the API usage is corrected with different parameter order we may have you moving forward?

This issue is closed now but experimental results are the real final say and reopening is just a button click away. Please let us know

Thanks!

Linkle123 commented 8 months ago

Oh my god I feel so stupid. It works now. Sorry to waste your time

david-allison commented 8 months ago

No worries! Happens to us all, glad we got you unblocked

mikehardy commented 8 months ago

@Linkle123 definitely happens to all of us, and I have to say that even though I didn't particularly want to spend a bunch of time getting the API resuscitated, it really needed it, and having someone (that would be you!) that I knew was testing stuff to validate the work as it happened really motivated me. The API was previously not building for distribution at all, had a crash immediately on startup, and had a useless error message for your case.

All that's fixed now. I'm going to call that a big win. Good luck with your project

Linkle123 commented 8 months ago

Thanks