getsentry / sentry

Developer-first error tracking and performance monitoring
https://sentry.io
Other
38.54k stars 4.12k forks source link

Minidump preprocess #29736

Closed mq1n closed 2 years ago

mq1n commented 2 years ago

Problem Statement

I looking for a solution for minidump file preprocessing before than Sentry analysis. My monitored application protected by some software protectors like VMProtect or Themida, these applications encrypt the application's stack information for protection therefore In case of a crash that may occur, the stack address information is shaped according to its encrypted form. In short, for this reason I must fix the application's minidump file before than analysis, so I must execute a process for doing minidump preprocessing. I am looking some solutions for this.

Solution Brainstorm

No response

getsentry-release commented 2 years ago

Routing to @getsentry/owners-native for triage. ⏲️

chadwhitacre commented 2 years ago

@mq1n Let's see if @getsentry/owners-native have any ideas on this one. Bringing into sentry since it is not really specific to self-hosting (unless I'm misunderstanding?).

Swatinem commented 2 years ago

We don’t support this in the pipeline directly, and I’m not sure we can add this with reasonably low effort.

Also, just by reading the marketing spiel of VMProtect, they say that:

the encrypted code runs on a virtual CPU that is markedly different from generic x86 and x64 CPUs as the command set is different for each protected file.

I’m wondering if the stack layout is even comparable to that of x64, and how its equivalent to call and ret would work. On x64, those would write/read instruction addresses directly to/from the stack.

mq1n commented 2 years ago

I'm not sure internally how they're doing, but the patching part is done with an .exe application given to customers by VMProtect,

Here: https://vmpsoft.com/20111114/minidump-fixer/

Swatinem commented 2 years ago

So that tool would probably be distributed alongside the application? Since we will definitely not be running this as part of our processing pipeline.

Also note the comment on that page:

you can’t debug crashes in the protected code

Which SDK are you using? Maybe its possible to put some code into crashpad that will shell out to that executable before doing the actual minidump upload.

mq1n commented 2 years ago

To explain briefly; this application, when run with the minidump file and the unprotected application, changes the minidump file stack informations according to the unprotected application's stack.

As you can guess the reasons, unfortunately it is not possible for me to distribute the unprotected version of the application and fixer tool to the clients, so I should to do it on the server side through a pipeline.

you can’t debug crashes in the protected code

That's meant for protected application debugging is not allowed for fixed minidump file, it's still requiring unprotected application.

Which SDK are you using?

I'm using the latest native SDK(0.4.2) with crashpad backend, but as I explained, it is not possible to do this through the client. Maybe as a last attempt, I can try to receive requests through a proxy server and process the minidump file and then pass it to the Sentry.

Swatinem commented 2 years ago

Ah okay, that all makes sense.

I can try to receive requests through a proxy server and process the minidump file and then pass it to the Sentry.

I think that is a feasible solution. Like I said, we won’t run some unverified code as part of our pipeline.

flub commented 2 years ago

Since we will definitely not be running this as part of our processing pipeline.

This isn't entirely true, we've talked about using webassembly to safely do this. But it's not even sure if or when we'd be building that feature for now.

However since this seems like a fixed binary you receive from an external vendor I assume being able to run webassembly in our processing pipeline would not help as you would not be able to compile it to webassembly?