The proguard mapping file size exceeded the maximum 50 MB size

[mahdi13]

Thanks for taking the time for reporting an issue!

Describe what happened

> The proguard mapping file at: [.../mapping.txt] size exceeded the maximum 50 MB size. This task cannot be performed.

The limit is not enough for many use cases, is there any way to upload larger mapping files?

Steps to reproduce the issue: Run the Gradle command:

:app:uploadMappingRelease

Describe what you expected: Being able to upload larger mapping files

Additional context

• Android Build Tools version: 30.0.2
• Gradle version: 7.0.2
• OS version: Ubuntu 20.04
• Plugin version:
• Proguard configuration:
• Other Gradle Plugins:

[xgouchet]

Hi @mahdi13, unfortunately this size limit is applied to all our uploading mecanism, and is not one we can overcome easily. Can you share with us your proguard configuration? Most importantly do you have the minifyEnabled set to true in your release configuration?

[muratke]

Hi @xgouchet,

this problem started to affect us as well. We're not able to upload the mappings which can make Datadog almost unusable for our android app. minifyEnabled is set to true in our release configuration.

[xgouchet]

Hi @muratke , unfortunately this is a hardcoded limit in our servers that we can't bypass right now. We do have a plan to try and handle larger mapping files but we don't have any ETA yet. How large are you mapping files?

[muratke]

@xgouchet It depends on the build but started to be 50-60 MBs. Do you have a workaround until you'd be able to increase the limit?

[xgouchet]

The only workaround that could work would be to disable obfuscation for all the third party open source libraries you're using. Because those are open source, you won't gain anything by obfuscating them. Doing so will limit the size of your mapping file.

[muratke]

While it is decreasing the mapping file size, it will increase the APK size and will not optimize/shrink the code. Might be used as a temporary solution, would like to get it fixed soon.

[xgouchet]

you can setup your Proguard file to still shrink parts of the code without obfuscating it. Using the keepnames instruction, you can specify classes and class members whose names are to be preserved, if they aren't removed in the shrinking phase.

[geranzo]

Hi, We hit the same problem and our mapping.txt file is close to 100 MB. However it compresses very well, down to around 5-6% of the original size. If the server limit is hard to change then perhaps this could be the way (modifying uploadMappingRelease task and your server-side end to accept compressed files) ?

[trymnilsen]

We are able to shave of a little bit but not enough to get below 50mb for the mapping file for our project. We have started using logging, but it would be nice to have crash reports as well in datadog. This is a blocker for us at the moment to transition.

[orafaaraujo]

Hello,

I'm facing the same issue.

Is possible to upload the file directly to the Datadog dashboard somehow? Although it's not scalable it can be a temporary solution while we work on our proguard file rules

Thanks

[xgouchet]

Hi @orafaaraujo, unfortunately the limit is not in the plugin itself but in our storage, so there'sno way for now to have a mapping file larger than 50 Mb. Since version 1.4.0, of the plugin, we do offer a renaming feature that allows you to shrink the size of the mapping file:

datadog {
  mappingFilePackageAliases = ["com.google.gson": "gson"]
  mappingFileTrimIndents = true
}

We're thinking on solutions we could use to support larger files but don't have any precise timeline to disclose yet.

[orafaaraujo]

Hey, @xgouchet

I'll review our R8 rules and apply your suggestion too, thank you for the quick reply!

[lalunamel]

Heyo - want to add our hat in the ring here. I'm the mobile infrastructure engineer at Gusto and we're also running into this issue. Our mapping file is 97mb, which as far as I've read elsewhere on the internet is not uncommon for any established kotlin app.

I tried out the packageAliases and trimIndents suggestion above in our own build and the space savings is marginal, unfortunately.

Would love to see a solution here that allows us to both use minification (because it reduces our bundle size and attracts more downloads) as well as error reporting (because error reporting is required to support users and build an app that is bug-free).

Edit: FWIW our previous logging and error reporting provider did this: bugsnag

[gyulavoros]

@lalunamel It actually seems like a really reasonable idea. Our mapping.txt file weighs around 63MB at the moment and compressing it into a ZIP archive, compress it down to 6.6 MBs.

@xgouchet Would something like this work on your end? Compress the mapping file, upload the archive, and extract it on the other end. Or is the problem still the underlying storage abstraction, which prevents files from exceeding 50 MBs?

[xgouchet]

hi @gyulavoros, there are many implication that we need to handle, but this is a topic we're actively investigating, to ensure that the Error Tracking experience works for applications with large mapping files

[Krupson]

@xgouchet any updates on the problem? What about compression mentioned by @geranzo? My mapping file got compressed from 52 to 4 MB using built-in system zipper.

[xgouchet]

Hi @Krupson, this is a topic we are working on, but is not a trivial task. We'll update this ticket as soon as we have something fixed.

[antondudakov]

same problem

[tevjef]

Here's a very crude solution I whipped up internally so we can get a basic demo of RUM. We basically just strip mappings we don't care about, for us, that means packages that don't match our app package. Obviously, this isn't very useful for crashes :/, we use already Firebase for that.

https://gist.github.com/tevjef/ce1e5d66ff800a43011ac1fc1a72b893

[MohammadRezaei92]

Here's a very crude solution I whipped up internally so we can get a basic demo of RUM. We basically just strip mappings we don't care about, for us, that means packages that don't match our app package. Obviously, this isn't very useful for crashes :/, we use already Firebase for that.

https://gist.github.com/tevjef/ce1e5d66ff800a43011ac1fc1a72b893

Where should I put ProguardMappingMapperTask file? Is any dependency needed?

[Krupson]

@MohammadRezaei92 I put it inside my buildSrc project. It's not default - you have to add it on your own.

[jasonxfhu]

I've also opted to trim the mapping file based on package, but during the uploadMappingRelease task, Datadog is doing some "optimizations" on the file and actually ends up making it 4x as big 🙃

Size of /build/outputs/mapping/release/mapping-striped.txt before optimization is 16014509 bytes
Size of optimized file is 62495216 bytes

[pvegh]

Same problem here with mapping.txt being 88MB

[0xnm]

We have published version 1.7.0 of our plugin, which bumps upload limit to 100 MB (only for US1 for now) and adds GZIP support for the faster upload.

[0xnm]

@jasonxfhu This is quite strange. Can you please share some block from the original file and optimized (few lines should be enough), so that we can understand why this happens?

[pvegh]

Nice. When can we count with this in the EU?

[0xnm]

@pvegh We hope that we will be able to relax the limits for other DCs in the few months, around the spring.

[MalcolmMcFly]

@0xnm is the following still valid in 1.7.0?:

datadog {
    mappingFileTrimIndents = true
    mappingFilePackageAliases = mapOf(
        "kotlinx.coroutines" to "kx.cor",
        "com.google.android.material" to "material",
        "com.google.gson" to "gson"
    )
}

[0xnm]

Hello @MalcolmMcFly! Yes, this API is still there. Do you experience any issues?

[tchang-acorns]

@0xnm Our mapping file is 154mb and even with these renaming features, it is still at 149mb. Is there any way for me to upload the zipped file itself?

[0xnm]

@tchang-acorns No, currently there is no such possibility (limits are checked for the uncompressed data). We are working on the improvements, I hope we will be able to make them available in Q2.

[MalcolmMcFly]

Here's a very crude solution I whipped up internally so we can get a basic demo of RUM. We basically just strip mappings we don't care about, for us, that means packages that don't match our app package. Obviously, this isn't very useful for crashes :/, we use already Firebase for that.

https://gist.github.com/tevjef/ce1e5d66ff800a43011ac1fc1a72b893

Can confirm this worked for us.

[Oliver1123]

We also experience this issue,

@pvegh We hope that we will be able to relax the limits for other DCs in the few months, around the spring.

0xnm any update on limitation changes for EU?

[0xnm]

@Oliver1123 We are hoping to have it by the end of Q2 or in Q3.

[0xnm]

Good news: we now support uploads up to 200 MB for US1 and EU1 sites, please update our Gradle Plugin to version 1.9.0 to be able to do such uploads.

US3, US5, AP1 still stay with 50 MB limit, but we are working on bringing the support for big uploads also there.

[jeremygriffes]

It looks like the upload size is still restricted to 50 MB for US1_FED. Can you please confirm?

[0xnm]

Hello @jeremygriffes! Yes, US1_FED still has 50 MB limit, we are working on solving it.

[0xnm]

We now support uploads up to 300 MB in US1, EU1, US3, US5. AP1 and US1_FED still have 50 MB limit.

[davidokonji]

Hello @0xnm Curious about how the zone is determined. is it using related to the DATADOG_SITE set? because currently using in the EU zone, but still getting the limit error size exceeded the maximum 50 MB size . Am I missing something?

[0xnm]

Hello @davidokonji! Yes, it is based on the DATADOG_SITE. Which version of the Gradle Plugin are you using? In order to be able to support big uploads, you need to use plugin version 1.9.0 or above.

[davidokonji]

Hi @0xnm I can confirm, the project uses distribution https\://services.gradle.org/distributions/gradle-8.0.1-all.zip

[0xnm]

@davidokonji I mean the version of our Datadog Gradle Plugin used, the full list of versions available can be found here https://github.com/DataDog/dd-sdk-android-gradle-plugin/releases.

[davidokonji]

Thanks for clarifying @0xnm , we are not using dd-sdk-android-gradle-plugin directly, but using it with expo-datadog@^50.0.0, @datadog/mobile-react-native@^1.8.4" and @datadog/datadog-ci@^2.26.0

[louiszawadzki]

Hi @davidokonji,

Indeed we set the value used in the expo plugin to be 1.5.+ (link to file):

      `    id("com.datadoghq.dd-sdk-android-gradle-plugin") version "1.5.+"`,

I'll make the change so that we use a more recent version, in the meantime and to get unblocked you can use patch-package to set the expo plugin to use "1.9.0" instead.

Let us know if that will solve your use case, we'll send a message here once the expo-plugin is released with the fix :)

[davidokonji]

@louiszawadzki @0xnm Thank you, I have applied the patch and I can confirm it works fine after that patch, looking forward to the new release.

[0xnm]

Closing this issue, because it is not relevant anymore, we support big file uploads now in all DCs.

[epool]

hey @0xnm 👋🏼 first of all thanks for all your support on this issue, but I'm still facing this issue with my mapping.txt file of 133.4 MB:

* What went wrong:
Execution failed for task ':app:uploadMappingRelease'.
> Unable to upload mapping file for `service:...`, `version:...`, `version_code:...`, `variant:`, `build_id:...` because mapping file is too large; please refer to documentation regarding the limits

with this setup

datadog_plugin_version = "1.14.0"
datadog = { id = "com.datadoghq.dd-sdk-android-gradle-plugin", version.ref = "datadog_plugin_version" }

plugins {
    alias(libs.plugins.datadog)
}

datadog {
    site = "US1"
}

it seems like it's falling in this condition

https://github.com/DataDog/dd-sdk-android-gradle-plugin/blob/fe2c9b6120902d6683eb7c7aedf99b8a0a08fa6a/dd-sdk-android-gradle-plugin/src/main/kotlin/com/datadog/gradle/plugin/internal/OkHttpUploader.kt#L183L186

could you please give me a hand with this one? 🙏🏼