Compatibility Issues with Sentry Java (Android, NDK) and Breakpad Backend

[bbtarzan12]

Problem Statement

Hello,

I am encountering issues in an environment that utilizes both Sentry Java (for Android, NDK) and Sentry Native. Specifically, when Sentry Java is used with its default configuration on Android, it defaults to using the inproc backend. This results in crash call stacks that only contain information from the crashing thread. Given the importance of obtaining call stack information from all threads upon a crash, I am considering using the breakpad backend to leverage minidumps.

Issues Encountered:

Inconsistency in Minidump Information: Repeatedly triggering the same crash and inspecting it via the Sentry web interface results in a random generation of events with and without minidump information. The cause of this inconsistency remains unidentified.

1. Contradictory Tag Display: Events containing minidump information lack Android-related tags, whereas events without minidump information display all Android-related tags.

2. Compatibility Inquiry: I am curious whether the combination of Sentry Java (Android, NDK) and the breakpad backend with Sentry Native is officially supported. Additionally, I seek solutions or workarounds for the issues mentioned above.

Additional Information:

• Envelope files generated on Android include an attachment of type event.minidump.
• The combination of Sentry Java and Breakpad Native results in discrepancies in the display of Android device-related tags.

These issues are hindering the analysis of crashes in our Android application. I would appreciate it if the Sentry team could verify these concerns and provide guidance or solutions.

Thank you.

---

Without minidump information

Missing Android Tag

[adinauer]

@markushi can you please take a look at this one?

[markushi]

@supervacuus maybe you can help me out here regarding 1: I guess tags are missing when using breakpad, because the minidumps are sent by breakpad itself, rather via the Android SDK which enriches the envelopes, right?

[bbtarzan12]

Still learning Sentry Java here. Does enableScopeSync help sync tags between native and Java layers? If that's the case, I've already set the enableScopeSync option to true. However, it seems like the tags aren't syncing up as expected. If there’s any more information I should provide, please let me know!

[supervacuus]

1: I guess tags are missing when using breakpad, because the minidumps are sent by breakpad itself, rather via the Android SDK which enriches the envelopes, right?

It is unclear how sentry-native was built in that context, so I don't know which configuration we're dealing with (@bbtarzan12, can you tell us which aspects of the build you adapted to enable the breakpad backend in sentry-android?). The breakpad backend relies on a transport being enabled in the Native SDK because breakpad itself has no transport mechanism.

If breakpad was explicitly specified as a backend, by default, the transport on Android would still be disabled. In that sense, the situation would be similar to using the inproc backend (i.e., envelopes will be serialized to disk and can be retrieved from there on the next start). The biggest difference is that it reads the generated minidump from disk to add it to the envelope file, so I think that the envelope will be correctly moved to the outbox (under the assumption that only the backend build parameter was changed). I could imagine that the outbox sender might have some assumptions regarding either the envelope items type or the event itself? Or maybe the backend has issues processing the minidump in the android crash context and association between event and minidump fails?

Even considering the above, this configuration is currently unsupported even if the backend configuration builds successfully because none of the paths from initialization to the envelope transport/processing have been tested for Android (it will be built like targeting Linux). In addition to that there is not client-symbolication in breakpad, which means that all device-specific symbols (libart, etc) won't be resolved. Breakpad also does not provide thread-names at this point, which especially with Android will be a considerable inconvenience.

CC: @kahest

[bbtarzan12]

I configured sentry-native project to use the breakpad backend by simply setting SENTRY_BACKEND to breakpad in the cmake configuration and proceeded with the build.

After a crash occurs, minidump and envelope files are correctly generated by breakpad and sentry. However, these files do not contain tags for Android, which is a crucial piece of information missing for my analysis.

I'm wondering if uploading separate debug symbols would enable support for thread names or symbolication? I'm aware that other solutions, such as App Center or Crashlytics, have successfully utilized breakpad for symbolication along with multiple threads.

Our project is developing an app that combines Android with native code, hence the need for detailed crash information on the native side.

@supervacuus

[bbtarzan12]

I'm curious to know whether this issue is unique to our project, or if the compatibility between breakpad and Android is not supported by design. If there is a potential fix, I would be grateful for any information you could provide. Thank you in advance

[supervacuus]

I'm curious to know whether this issue is unique to our project, or if the compatibility between breakpad and Android is not supported by design.

This issue is not specific to your project, breakpad, or Android. The handling of minidump envelopes is currently unsupported in sentry-android. There is no roadmap to add support.

I configured sentry-native project to use the breakpad backend by simply setting SENTRY_BACKEND to breakpad in the cmake configuration and proceeded with the build.

Thanks, that is what I expected. Under this assumption, I wrote my response above.

After a crash occurs, minidump and envelope files are correctly generated by breakpad and sentry. However, these files do not contain tags for Android, which is a crucial piece of information missing for my analysis.

Yes, again, there is currently no support for the combination of these envelopes in sentry-android. Even if the Native SDK with the breakpad backend produces the correct envelopes on Android, they are currently not correctly sent from the OutboxSender in sentry-android because that use-case is not implemented or tested in the Native SDK integration.

I'm wondering if uploading separate debug symbols would enable support for thread names or symbolication?

You can only upload debug symbols for files you own, but this is not generally true for all devices your app deploys to (except if you have limited deployment to devices you own). This is why client symbolication is required on Android. We have client symbolication in the inproc and crashpad backend (the latter is also not supported on Android).

Our project is developing an app that combines Android with native code, hence the need for detailed crash information on the native side.

We know that stack traces for more than the crashing thread are helpful for error analysis, and multiple approaches exist to achieve this. Sentry is working on providing a solution, but whether adding breakpad support is one of these is still being determined.

cc: @kahest, @markushi