search

DataDog / dd-sdk-android-gradle-plugin

The Datadog Gradle Plugin for Android
Apache License 2.0
14 stars 9 forks source link

Stracktraces obfscuated despite mapping files being uploaded #283

Closed mformetal closed 2 weeks ago

mformetal commented 4 months ago

Thanks for taking the time for reporting an issue!

Describe what happened

We're using this plugin to upload mapping files to de-obfsucate our stacktraces. I can validate that these mapping files were successfully uploaded via gradle output and when I view the RUM Debug Symbols page, but our traces are still garbled.

Steps to reproduce the issue:

We build a release application, release it, and expect to see traces on RUM de-obsfucated.

Describe what you expected:

Non-obfsucated stacktraces.

Additional context

-keepattributes Annotation -keepattributes SourceFile,LineNumberTable -keep public class * extends java.lang.Exception

GeoComply Proguard rules

-keep class com.geocomply. { *; } -keep class com.google.* { ; } -keep class android.support. { *; }

---------------Begin: proguard configuration for Gson ----------

Gson uses generic type information stored in a class file when working with fields. Proguard

removes such information by default, so configure it to keep all of it.

-keepattributes Signature

Gson specific classes

-dontwarn sun.misc.**

Prevent proguard from stripping interface information from TypeAdapter, TypeAdapterFactory,

JsonSerializer, JsonDeserializer instances (so they can be used in @JsonAdapter)

-keep class implements com.google.gson.TypeAdapter -keep class implements com.google.gson.TypeAdapterFactory -keep class implements com.google.gson.JsonSerializer -keep class implements com.google.gson.JsonDeserializer

Prevent R8 from leaving Data object members always null

-keepclassmembers,allowobfuscation class * { @com.google.gson.annotations.SerializedName ; }

---------------End: proguard configuration for Gson ----------

-assumenosideeffects class android.util.Log { public static d(...); public static v(...); }

Navigation

-keep public class * implements android.os.Parcelable

-dontwarn org.conscrypt.Conscrypt$Version -dontwarn org.conscrypt.Conscrypt -dontwarn org.conscrypt.ConscryptHostnameVerifier -dontwarn org.openjsse.javax.net.ssl.SSLParameters -dontwarn org.openjsse.javax.net.ssl.SSLSocket -dontwarn org.openjsse.net.ssl.OpenJSSE

-keepclassmembernames class bet.thescore.android.ui.webview.WebViewEvent$RunLexisNexisProfiling { *; }

-dontwarn com.miui.referrer. -dontwarn com.samsung.android.sdk.sinstallreferrer.

Keep public ViewModel constructors

-keepclassmembers public class * extends androidx.lifecycle.ViewModel { public (...); }


 - Other Gradle Plugins:

org.gradle.testing.jacoco.plugins.JacocoPlugin io.gitlab.arturbosch.detekt.DetektPlugin com.google.firebase.crashlytics.buildtools.gradle.CrashlyticsPlugin com.google.firebase.perf.plugin.FirebasePerfPlugin org.gradle.api.plugins.JvmToolchainsPlugin com.android.build.gradle.AppPlugin com.google.gms.googleservices.GoogleServicesPlugin androidx.navigation.safeargs.gradle.SafeArgsJavaPlugin com.datadog.gradle.plugin.DdAndroidGradlePlugin io.github.takahirom.roborazzi.RoborazziPlugin org.jetbrains.kotlin.gradle.internal.Kapt3GradleSubplugin org.jetbrains.kotlin.gradle.plugin.KotlinAndroidPluginWrapper com.adarshr.gradle.testlogger.TestLoggerPlugin org.jetbrains.kotlin.gradle.internal.ParcelizeSubplugin app.cash.paparazzi.gradle.PaparazziPlugin

0xnm commented 3 months ago

Hello @mformetal! Sorry for the very late reply.

Can you please share which version of our SDK you are using? If it is the version above 2.9.0, can you please check that error events you see in RUM/Logs have a build_id attribute with a value equal to the value shown in the mapping file upload?

RSM-CZE commented 2 months ago

Hey!

I am currently running into the same problem as described here. However, a few times the mapping was successful.

To give more detail (for when it does not work): SDK Version: 2.13.1 Plugin Version: 1.14.0

The build_id attribute found in the error-log-JSON does not match any uploaded build_id from the uploaded mapping files. Seems logical, that matching is not possible here.

We upload the symbols right after minifying on CI:

afterEvaluate {
  if (System.getenv("DATADOG_API_KEY").isNullOrBlank()) return@afterEvaluate
    tasks.named("minify{...}WithR8") {
      finalizedBy("uploadMapping{...}")
    }
}

Logs show everything happening in the correct sequence, with the app built and distributed as an APK in our test environment:

> Task :app:uploadMapping{...}
Mapping file upload successful with tags `service:{...}`, `version:X.Y.Z`, `version_code:XXXXX`, `variant:`, `build_id:XXXXXXXXX`. 
Make sure the SDK is initialized with the same values to ensure proper deobfuscation. Mapping files will be processed and ready within the next five minutes.

> Task :app:compile{...}ArtProfile
> Task :app:package{...}
> Task :app:create{...}ApkListingFileRedirect
> Task :app:assemble{...}

> <___Tasks that upload APK from outputs/___>

From checking out the source code of the plugin and SDK, it seems that the build_id is just read from file inside of the SDKs initialisation process, so as you would expect, we do not set, manipulate or use the file at all in our code.

Do you have any insight or idea why this could be happening? Why are the build_ids not the same? Greatly appreciate the time!

0xnm commented 2 months ago

Hello @RSM-CZE! Build ID is indeed injected by the Datadog Gradle Plugin.

The reason it may be different is having APK produced in the different build run than mapping file upload. Then in this case you may have different Build IDs for these two separate build runs. May it happen in our case?

Do you mind also to share the AGP version you are using?

RSM-CZE commented 2 months ago

I thought of that as well, but this is the only pipeline that uploads the mappings (because it uses the only machine that has the API_KEY set). There are no new assemble tasks in between the mapping upload and the APK upload.

I can also reproduce this behaviour when building locally.

AGP: 8.3.2, Gradle: 8.9

0xnm commented 2 months ago

@RSM-CZE Is it possible for you to provide a minimal reproducible example project?

RSM-CZE commented 2 months ago

I've been unable to reproduce the issue in a minimal project, but I've identified the main symptom in our primary project. It seems the buildId is somehow cached/ not updated. After the first build in a clean environment, the buildId remains unchanged in the APK assets after running any new ./gradlew clean assembleXYZ tasks, even though it updates in the build folder and for the uploaded mappings.

I have enough information to implement a temporary workaround and will revisit this issue when I can prioritise it again.

Bit more insight to the first report: On CI, we assemble 2 times in a row: Once for checklist/tests and a second time for distribution. This happens on the same machine, but mappings are only uploaded once (on distribution). However, as described above, the buildId does not change in the newly generated APK. Therefore we upload mappings for the second generated buildId, but the second generated APK still has the first generated buildId.

0xnm commented 2 months ago

@RSM-CZE This is kind of strange, because buildId should be updated.

Can you please share what are the command/tasks/inputs participating in the first build (checklist/tests as you've described), and what are the same values for the second build?

Also can you please share a workaround you've applied?

Maybe updating AGP version can solve your issue?

RSM-CZE commented 1 month ago

Hi @0xnm,

Apologies for the delayed response!

Updating AGP to 8.6.1 resolved the issue. Although I further tested isolated scenarios in a minimal project, I couldn't replicate the exact behaviour seen in the main project, so I can't provide further details on the root cause.

Thank you for your quick and helpful responses! 😊