Using Unity as a Library does not report crashes/fatals

Esildor commented 4 months ago

Environment

How do you use Sentry? Sentry SaaS

Which version of the SDK? 2.1.0

How did you install the package? Git-URL

Which version of Unity? 6000.0.4f1

Is this happening in Unity (editor) or on a player like Android, iOS, Windows? Android

Steps to Reproduce

Download and open the Unity as a Library (UaaL) example.
Install Sentry as a package.
Run the setup wizard to ensure the Sentry plugin is activated.
Ensure that Android Native Support is enabled.
Add this script this script to the project. This will add buttons to simulate various crashes via UnityEngine.Diagnostics.Utils.ForceCrash(). To the top left of the screen.
Continue setting up the UaaL project per the instructions in Unity's Github docs
Build and run on a device
Press any of the ForcedCrashCategory buttons, for example: Engine Crash - FatalError, Engine Crash - AccessViolation, etc.

Expected Result

Unity engine-level crashes/fatal issues are reported to Sentry.

Actual Result

No fatal crash is reported. It's also worth noting: Performing these steps in a non-UaaL project reports fatal level crashes to Sentry as expected, see the image below.

Any logs or screenshots

bitsandfoxes commented 4 months ago

Thanks @Esildor for reaching out! Tbh I've never tried the UaaL sample. I'll get back to you after giving this a try!

bitsandfoxes commented 3 months ago

This has not been forgotten. I'm working my way through the guide.

Esildor commented 3 months ago

Thanks for the update 💖

bitsandfoxes commented 1 month ago

We're working on, well, making that work. There are a couple of questions that need to be answered first:

How (if) the Unity SDK should sync with the surrounding native SDK
Do those SDK share a DSN? Should the Unity SDK just send events one way and not care?
Should the scope be de-synced during the game's shutdown. I.e. The native app is some sort of launcher and there are a bunch of Unity games

bitsandfoxes commented 1 month ago

I've created a UaaL repro here to have something to work off of.

bitsandfoxes commented 1 month ago

Context

When exporting as a library Unity we have a couple of different things to watch out for:

The Unity game itself is covered by the Unity SDK
Native Android Support should be disabled as to not create conflicts with the surrounding app
The surrounding app brings its own Android SDK that is responsible to capturing native error

Setup for Automatic Native Crash Capture

Number 3 is currently the crux. When exporting as a library, Unity has its own activity to run in

<activity
    android:name=".MainUnityGameActivity"
    android:configChanges="mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|uiMode|screenSize|smallestScreenSize|fontScale|layoutDirection|density"
    android:hardwareAccelerated="false"
    android:process=":Unity"
    android:screenOrientation="fullSensor"></activity>

To get the Android SDK to have the NDK integration running in the separate process we have to follow our "multi process apps" docs and add

<provider
    android:name="io.sentry.android.core.SentryInitProvider"
    android:authorities="${applicationId}.SentryInitProvider"
    android:exported="false"
    android:process=":Unity"
    tools:node="merge"
/>

to the AndroidManifest.xml The process name can be taken directly from the activity.

Gotcha

Add uploadNativeSymbols = true and have the thing set to release in the build variant.

Result

Having followed this setup native crashes do automatically get captured. But the stacktrace does not look too good.

Open Issues

There are a couple issues remaining, mostly debug symbols and IL2CPP related.

Debug Symbol Upload

The debug symbols automatically uploaded by the Android Gradle Plugin seem to be lacking the debug companions. Uploading the symbols manually after the build via sentry-cli debug-files upload ../uaal-unity/UnityProject/Library/Bee --org sentry-sdks --project sentry-unity fixes that and improves the stacktrace quality.

C# line numbers and IL2CPP Mappings

Because native support during the export is disabled we also don't generate and upload the IL2CPP mappings errors captured by the Unity SDK are missing line numbers. but this can be fixed on the SDK side.

C# frames in native crashes

The screenshots of native crashes have a couple of __emutls_get_address frames. Looking at the symbolicated C# exception we can see these as symbols. It looks like we need to find a way to get from the symbols to the actual function during symbolication?

"stacktrace": {
          "frames": [
            {
              "function": "ThrowNull",
              "symbol": "__emutls_get_address",
              "module": "BugFarmButtons",
              "package": "Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null",
              "in_app": true,
              "data": {
                "symbolicator_status": "symbolicated"
              },
              "instruction_addr": "0x8000000000fd77b0"
            }
          ],
          "instruction_addr_adjustment": "none"

TLDR; What to do?

Improve automatic debug symbol upload when exporting
- We could either simply upload during export. But I'm not sure if these symbols stay valid after the Android build. Maybe have an additional flag for "symbol upload during export" and we modify the UnityLib gradle file like we would in a regular export.
Fix line numbers in C# error. But that relies on us having a gameplan for 1.
Improve stacktrace quality

bitsandfoxes commented 1 week ago

Step by step guide to capture native crashes via the surrounding Android SDK - NO Sentry Unity SDK

1. Export the Unity project.

You don't need the Unity SDK at this point.

2. Run the wizard out the outputted Android project.

I chose MainApp as target. This will modify the build.gradle here

plugins {
    id 'io.sentry.android.gradle' version '4.13.0'
}

and here, adding the sentry-cli options.

The wizard will also add the options to the AndroidManifest.xml here.

3. Add a custom `Provider` to the `AndroidManifest.xml`

By default - and following the UaaL sample - the Unity game will run in its own process. Grab the process' name i.e. from here android:process=":Unity" and add it to the AndroidManifest.xml

<provider
    android:name="io.sentry.android.core.SentryInitProvider"
    android:authorities="${applicationId}.SentryInitProvider"
    android:exported="false"
    android:process=":Unity"
    tools:node="merge"
/>

4. Update the sentry-cli options to upload native symbols (optional - if your app uses native plugins?)

You can need to opt-in the native symbol upload by adding the relevant flags here

uploadNativeSymbols = true
autoUploadNativeSymbols = true

5. Run `sentry-cli` on the root

The Android Gradle Plugin has difficulties dealing with those modular applications so even adding it to the unityLibrary did not work.

Creating a sentry.properties on the root and running

sentry-cli debug-files upload --org *insert org* --project *insert-project*

on the project's root uploads required the symbols to symbolicate those native issues.

It's not necessary to run `sentry-cli` on the Unity build-cache

When running tests I attempted to pick up any missing debug symbols from ./UnityProject/Library/Bee/artifacts/Android/ but running sentry-cli there after running it on the android project root resulted in

> Found 27 debug information files (2 with embedded sources)
> Prepared debug information files for upload
> Nothing to upload, all files are on the server

Result

Following those steps results in fully symbolicated stacktraces. This is a native error, from a native plugin from within a Unity game running in its own process within an Android app, captured by the Android app's Sentry Android SDK.

bitsandfoxes commented 1 week ago

Adding capturing of C# errors - Adding the Unity SDK

To capture in-game C# errors and have the proper support for Unity specific contexts you need to add the Sentry SDK for Unity. This will also enable IL2CPP line number support, giving you actual C# line numbers instead of the generated C++ ones.

Disable Android Native Support

To avoid dependency conflicts, since the SDK for Android will already be in the app, you need to opt-out of the Android Native Support in the Unity SDK.

Update the `sentry-cli` command to upload IL2CPP line number mappings as well

Adding the Unity SDK to your game will automatically generate the mappings during the build. To upload those mappings you need to update the command you used to upload debug symbols by adding --il2cpp-mapping like so

sentry-cli debug-files upload --il2cpp-mapping --org *insert org* --project *insert-project*

Result

Adding the steps above will make sure that the errors inside the game are properly caught and symbolicated and their stacktraces show proper C# line numbers.

Limitations

There are a couple of current limitations

No Unity contexts and breadcrumbs in native crashes

Since the Unity SDK has no control over the surrounding Android SDK and native support has been disabled the SDK will not attempt to sync scope or breadcrumbs. This leads to native crashes that originate from in-game lacking the in-game context and breadcrumbs

Calling `Close()` on the Unity SDK will also shut down the surrounding Android SDK

The SDK subscribes to the game's shutdown and closes itself with the game. It does this for the native SDK as well. A fix is on its way https://github.com/getsentry/sentry-unity/pull/1897

getsentry / sentry-unity