microsoft / mimalloc

mimalloc is a compact general purpose allocator with excellent performance.
MIT License
9.74k stars 793 forks source link

"minject.exe --inplace --force --postfix=override <exe>" intermittently crashes (v2.1.2) #850

Open LPhil opened 4 months ago

LPhil commented 4 months ago

We have the problem that minject.exe crashes sporadically during the build in our build environment (building with jekins, running a service in the background). The crash occurs in about 30% of the builds in this environment. When building as a user in the foreground (VS2022 with v143), the problem is not reproducible. In our environment there are about 5 unit tests where minject.exe crashes.

Question: Is there a way to start minject.exe to generate more debug output via \<options> to find out where and what causes the crash? The problem occurs in both the debug and the release environment.

In a BAD case, the output looks like this. minject.exe -v --inplace --force --postfix=override unit.test.exe reading 'unit.test.exe' inject 'mimalloc-redirect.dll' module 'mimalloc-override' is already imported leave at position 0: 'mimalloc-redirect.dll' leave at position 1: 'mimalloc-override.dll'

module order unchanged ==> Crash / exited with code -1073740940

In a GOOD case, the output looks like this. reading 'unit.test.exe' inject 'mimalloc-redirect.dll' module 'mimalloc-override' is already imported leave at position 0: 'mimalloc-redirect.dll' leave at position 1: 'mimalloc-override.dll'

module order unchanged original imported modules (36): 0: mimalloc-override.dll 1: KERNEL32.dll 2: MSVCP140.dll 3: VCRUNTIME140.dll 4: VCRUNTIME140_1.dll 5: api-ms-win-crt-runtime-l1-1-0.dll ... 32: WINTRUST.dll 33: CRYPT32.dll 34: NETAPI32.dll 35: WS2_32.dll

wrote (intermediate) 'unit.test-mi.exe' with new import order. wrote 'unit.test.exe' with new import order.

Thanks for your support

LPhil commented 4 months ago

We are using the latest dev-slice version of minject.exe because we had the problem under ticket #734.

What exactly are the differences between the two versions of minject.exe? Can the changes from 2531f5708bffc50eaf15383e676c8ebe8a3d9ac5 be merged into the main branch with the next release?

LPhil commented 4 months ago

Code 0xC0000374 (-1073740940) is STATUS_HEAP_CORRUPTION (A heap has been corrupted.)

Faulting application name: minject.exe, version: 0.0.0.0, time stamp: 0x62167569 Faulting module name: ntdll.dll, version: 10.0.19041.3996, time stamp: 0x39215800 Exception code: 0xc0000374 Fault offset: 0x00000000000ff349 Faulting process id: 0xef8 Faulting application start time: 0x01da6bd602dd37c4 Faulting application path: [PathToMinject]\minject.exe Faulting module path: C:\Windows\SYSTEM32\ntdll.dll ...

daanx commented 4 months ago

Hi -- thanks for the report; I understand that minject itself is crashing? Not the exe generated by minject ? Very strange. If it crashes, you cannot reproduce it as a normal user right? That is, if it crashed, it may just work if you try it manually?

the heap corruption error is strange too as it does not allocate much, .. but I guess there is a buffer overflow in minject itself (which would also explain why it only crashes sometimes as it would depend on the heap layout). If you can repro on a certain binary that would be great. I will look into it as well but it is hard to find a bug without repro.. tbc

LPhil commented 2 months ago

Hey, sorry for the long delay.

I understand that minject itself is crashing? Not the exe generated by minject ? Correct, the minject.exe crashes during the reordering process.

If it crashes, you cannot reproduce it as a normal user right? Correct, minject.exe only crashes when it is called with Jenkins (java) background process. I have never been able to reproduce the problem in a native Windows environment.

That is, if it crashed, it may just work if you try it manually? Correct, if I call minject.exe with the same parameters with the exe that led the problem in the background, it does not cause any problems.

The crash occurs sporadically with the same 5 exe files, as I said it cannot be reproduced manually.

@daanx: Did your analye reveal anything?