getsentry / sentry-native

Sentry SDK for C, C++ and native applications.
MIT License
390 stars 165 forks source link

Show function parameters and/or local variables? #353

Open mikepurvis opened 4 years ago

mikepurvis commented 4 years ago

My company's current Sentry setup relies on grabbing full native core dumps and symbolicating them using gdb. Sentry is able to nicely render the output of this process with each stack frame having a fold-out for local variables— including even some expansion of structs and container types like vectors (basically, whatever gdb was able to achieve given its context and plugins). This is what a small test program looks like when symbolicated with gdb:

image

But going through crashpad->symbolicator, all that are captured are the registers:

image

As far as I can tell, the minidump format does permit including this level of detail, but either breakpad/crashpad aren't capturing it, or symbolicator/sentry aren't rendering it. A small test round-tripping a core through the core2md and minidump-2-core tools that come with breakpad shows gdb no longer able to resolve the memory addresses of the locals afterward.

Anyway, is there something I can change about to enable a fuller stack capture, or alternatively, is something like this being tracked as a feature ask anywhere?

Swatinem commented 4 years ago

is there something I can change about to enable a fuller stack capture

At least when using crashpad, you can use CrashpadInfo::set_gather_indirectly_referenced_memory to capture more parts of the memory. You can set that via crashpad independently from any sentry settings.

is something like this being tracked as a feature ask anywhere?

I don’t think we have this on our roadmap. Also, I don’t think we resolve any locals, even if you would have enough regions in the minidump to be able to.

mikepurvis commented 4 years ago

Interesting, thanks. That does look close to what I want. I naively added it to my test program:

crashpad::CrashpadInfo *crashpad_info = crashpad::CrashpadInfo::GetCrashpadInfo();
crashpad_info->set_gather_indirectly_referenced_memory(crashpad::TriState::kEnabled, 1000000);

Unsurprisingly the resulting dump looks the same in terms of what Symbolicator/Sentry renders. I will continue to play with this on my end, though, to better understand the size impacts, etc.

It would be nice if there was a way in Sentry to download the minidump for the purposes of locally rehydrating it into a full core and then digging into any of the extra memory regions saved (via this or set_extra_memory_ranges. I know you intentionally don't archive dumps on the hosted product because of the PII liability, but that is not a consideration in my application (robotics).

Swatinem commented 4 years ago

It would be nice if there was a way in Sentry to download the minidump

you can. It is at the bottom listed in the attachments section.

mikepurvis commented 4 years ago

I don't seem to have the attachments section on 20.8.0.dev0803866c; is there something I need to do to turn it on? I checked briefly on my /settings/sentry/security-and-privacy page, and didn't see anything that seemed applicable.

I assumed it was intentionally missing/disabled based on this verbiage:

Crash reports may contain sensitive information about the target system, such as environment variables, local pathnames, or in-memory representations of input fields, including passwords. By default, Sentry only uses crash report files to create events and immediately drops them. All sensitive information is stripped from the resulting events.

https://docs.sentry.io/enriching-error-data/attachments/#crash-reports-and-privacy

jan-auer commented 4 years ago

Indeed, attachments are disabled by default and have to be enabled in your server configuration. You can do this by adding "organizations:event-attachments" to the SENTRY_FEATURES list.

We will look into enabling this by default.