APP CRASH (win8x86 calc under DrMem) chkstk stack zeroing breaking transparency

[derekbruening]

From bruen...@google.com on December 13, 2013 21:32:00

win8 x86 VM: start calc under DrMem, and then click Help|About Calculator. (that works on win8.1 x64 VM).

Dr.M WARNING: application exited with abnormal code 0xc0000409

define STATUS_STACK_BUFFER_OVERRUN ((NTSTATUS)0xC0000409L) // winnt

bin/drmemory.exe -leaks_only -no_count_leaks -no_track_allocs -- calc

Dr.M WARNING: application exited with abnormal code 0xc0000409

These all work:

bin32/drrun -t drltrace -logdir . -- calc

bin32/drrun -debug -t drltrace -logdir . -- calc

bin32/drrun -debug -stderr_mask 15 -disable_traces -bb_single_restore_prefix -max_bb_instrs 256 -vm_size 256M -no_enable_reset -no_early_inject -t drltrace -logdir . -- calc

This works: -no_zero_retaddr

So it's a transparency issue with zeroing the retaddr.

in event_basic_block(tag=0x71c1bdf3) COMCTL32!CMarkup::Add+0x1c (returning) zero retaddr for normal ret in event_basic_block(tag=0x71d4903a) COMCTL32!CMarkup::_GetNextAnchorTag+0x12c60c in event_basic_block(tag=0x71c1cbaa) COMCTL32!CMarkup::_GetNextAnchorTag+0x178 in event_basic_block(tag=0x71c1cbb1) COMCTL32!CMarkup::_GetNextAnchorTag+0x17f in event_basic_block(tag=0x71c64408) COMCTL32!__report_gsfailure in event_basic_block(tag=0x71c644fd) in event_basic_block(tag=0x74bed5b4) system call #176==176.0 NtQueryInformationProcess

0 ntdll.dll!ZwQueryInformationProcess+0xa (0x772757ce <ntdll.dll+0x157ce>) modid:0

1 fp=0x004ea39c parent=0x004ea5dc KERNELBASE.dll!SetUnhandledExceptionFilter+0x84 (0x74be715c <KER

NELBASE.dll+0x2715c>) modid:0

2 fp=0x004ea5dc parent=0x004ea5e8 COMCTL32.dll!__raise_securityfailure+0xc (0x71c6450a <COMCTL32.d

ll+0x9450a>) modid:0

3 fp=0x004ea5e8 parent=0x004ea918 COMCTL32.dll!__report_gsfailure+0xe0 (0x71c644e9 <COMCTL32.dll+0x944e9>) modid:0

4 fp=0x004ea918 parent=0x004ebb94 COMCTL32.dll!CMarkup::_GetNextAnchorTag+0x199 (0x71c1cbc8 <COMCTL32.dll+0x4cbc8>) modid:0

0:000> U 0x71c1cbb1 L10 COMCTL32!CMarkup::_GetNextAnchorTag+0x17f: 71c1cbb1 8b8d9cedffff mov ecx,dword ptr [ebp-1264h] 71c1cbb7 8901 mov dword ptr [ecx],eax 71c1cbb9 8b4dfc mov ecx,dword ptr [ebp-4] 71c1cbbc 5f pop edi 71c1cbbd 8bc6 mov eax,esi 71c1cbbf 5e pop esi 71c1cbc0 33cd xor ecx,ebp 71c1cbc2 5b pop ebx 71c1cbc3 e848f00200 call COMCTL32!__security_check_cookie (71c4bc10) 71c1cbc8 c9 leave 71c1cbc9 c20c00 ret 0Ch

0:000> U 71c4bc10 COMCTL32!security_check_cookie: 71c4bc10 3b0de8b7d671 cmp ecx,dword ptr [COMCTL32!security_cookie (71d6b7e8)] 71c4bc16 0f85ec870100 jne COMCTL32!__report_gsfailure (71c64408) 71c4bc1c c3 ret

We have 3 zeroings: regular ret, SEH, and chkstk. I tracked this down to the chkstk zeroing.

Original issue: http://code.google.com/p/drmemory/issues/detail?id=1405

[derekbruening]

From bruen...@google.com on December 13, 2013 19:03:46

In particular, the 4th one, in comctl32:

% grep chkstk drmemory/logs/DrMemory-calc.exe.4016.000/global.4016.log found _chkstk at 0x003c6470 found _chkstk at 0x75263aec found _chkstk at 0x754e5892 found _chkstk at 0x71c2629d

COMCTL32!_alloca_probe: 71c2627c 51 push ecx 71c2627d 8d4c2404 lea ecx,[esp+4] 71c26281 2bc8 sub ecx,eax 71c26283 1bc0 sbb eax,eax 71c26285 f7d0 not eax 71c26287 23c8 and ecx,eax 71c26289 8bc4 mov eax,esp 71c2628b 2500f0ffff and eax,0FFFFF000h 71c26290 3bc8 cmp ecx,eax 71c26292 7309 jae COMCTL32!_alloca_probe+0x18 (71c2629d) 71c26294 2d00100000 sub eax,1000h 71c26299 8500 test dword ptr [eax],eax 71c2629b ebf3 jmp COMCTL32!_alloca_probe+0x14 (71c26290) 71c2629d 8bc1 mov eax,ecx <--- 71c2629f 59 pop ecx 71c262a0 94 xchg eax,esp 71c262a1 8b00 mov eax,dword ptr [eax] 71c262a3 890424 mov dword ptr [esp],eax 71c262a6 c3 ret

Fragment 52451, tag 0x71c2629d, flags 0x1000030, shared, size 45: [COMCTL32.dll~DSA_Sort+0x38d9,~DSA_GetItem-0x107c]

0:000> U 0x22bfb4c4 L200 22bfb4c4 648b0de80e0000 mov ecx,dword ptr fs:[0EE8h] 22bfb4cb 8bc1 mov eax,ecx 22bfb4cd 59 pop ecx 22bfb4ce 94 xchg eax,esp 22bfb4cf 8d10 lea edx,[eax] 22bfb4d1 8b00 mov eax,dword ptr [eax] 22bfb4d3 c70200000000 mov dword ptr [edx],0 22bfb4d9 890424 mov dword ptr [esp],eax 22bfb4dc 64890de80e0000 mov dword ptr fs:[0EE8h],ecx 22bfb4e3 59 pop ecx 22bfb4e4 c74424fc00000000 mov dword ptr [esp-4],0 22bfb4ec e90f6d4bff jmp 220b2200

called from here: 0:000> U 0x71c1bd25 COMCTL32!CMarkup::Parse: 71c1bd25 8bff mov edi,edi 71c1bd27 55 push ebp 71c1bd28 8bec mov ebp,esp 71c1bd2a b8bc100000 mov eax,10BCh 71c1bd2f e848a50000 call COMCTL32!_alloca_probe (71c2627c) 71c1bd34 a1e8b7d671 mov eax,dword ptr [COMCTL32!__security_cookie (71d6b7e8)]

        +0x000 edi              : 0x4eaaf8
        +0x004 esi              : 0x4cd29f4
        +0x008 ebp              : 0x26d258
        +0x00c esp              : 0x26d250
        +0x010 ebx              : 0
        +0x014 edx              : 0
        +0x018 ecx              : 0x26c198
        +0x01c eax              : 0x26c000
        +0x020 eflags           : 0x202

0:000> dds 0x26d250 0026d250 04cd29f4 0026d254 71c1bd34 COMCTL32!CMarkup::Parse+0xf 0026d258 0026d270 0026d25c 71c26434 COMCTL32!CMarkup::SetText+0x4f 0026d260 004eaaf8

The extra slot will go away with the pop.

[derekbruening]

From bruen...@google.com on December 13, 2013 20:17:54

00c7bce8 00c7bd08

Here's what the xor value should be: 0:000> ? bbafdd88 ^ 00c7bcdc Evaluate expression: -1150787244 = bb686154

Watchpoint -> we see it stored where we expect: 23e3e25d 8b4508 mov eax,dword ptr [ebp+8] ss:0023:00c7bce4=e8aad700 0:000> dds 00c7bcd8 L 3 00c7bcd8 bb686154 00c7bcdc 00c7cdb8 00c7bce0 71c1c772 COMCTL32!CMarkup::Parse+0xc2

And we see where it gets clobbered, b/c of edx pointing there: 0:000> g Breakpoint 1 hit eax=058cfb84 ebx=00d7aae8 ecx=00000000 edx=00c7bcd8 esi=80004005 edi=00000000 eip=23e3e29e esp=00c7aa5c ebp=00c7bcdc iopl=0 nv up ei pl zr na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 23e3e29e 50 push eax 0:000> dds 00c7bcd8 L 3 00c7bcd8 bb680000 00c7bcdc 00c7cdb8 00c7bce0 71c1c772 COMCTL32!CMarkup::Parse+0xc2 0:000> U 23e3e29b 23e3e29b 66890a mov word ptr [edx],cx

So edx is assumed to not be clobbered, in fact it must be passed in or sthg: COMCTL32!CMarkup::_GetNextAnchorTag: 71c1ca2e 8bff mov edi,edi 71c1ca30 55 push ebp 71c1ca31 8bec mov ebp,esp 71c1ca33 b874120000 mov eax,1274h 71c1ca38 e83f980000 call COMCTL32!_alloca_probe (71c2627c) 71c1ca3d a1e8b7d671 mov eax,dword ptr [COMCTL32!__security_cookie (71d6b7e8)] 71c1ca42 33c5 xor eax,ebp 71c1ca44 8945fc mov dword ptr [ebp-4],eax 71c1ca47 8b4508 mov eax,dword ptr [ebp+8] 71c1ca4a 53 push ebx 71c1ca4b 56 push esi 71c1ca4c 89858cedffff mov dword ptr [ebp-1274h],eax 71c1ca52 8b450c mov eax,dword ptr [ebp+0Ch] 71c1ca55 898d90edffff mov dword ptr [ebp-1270h],ecx 71c1ca5b 8b4d10 mov ecx,dword ptr [ebp+10h] 71c1ca5e 57 push edi 71c1ca5f 33ff xor edi,edi 71c1ca61 89859cedffff mov dword ptr [ebp-1264h],eax 71c1ca67 8b00 mov eax,dword ptr [eax] 71c1ca69 668939 mov word ptr [ecx],di 71c1ca6c 898d98edffff mov dword ptr [ebp-1268h],ecx 71c1ca72 be05400080 mov esi,80004005h 71c1ca77 33c9 xor ecx,ecx 71c1ca79 899594edffff mov dword ptr [ebp-126Ch],edx 71c1ca7f 89b5a8edffff mov dword ptr [ebp-1258h],esi 71c1ca85 66890a mov word ptr [edx],cx

It looks like it's part of some unknown calling convention that passes data in ecx and edx. Here's the caller: COMCTL32!CMarkup::Parse+0xa5: 71c1c755 8d8554efffff lea eax,[ebp-10ACh] 71c1c75b 50 push eax 71c1c75c 8d8550efffff lea eax,[ebp-10B0h] 71c1c762 50 push eax 71c1c763 53 push ebx 71c1c764 8d559c lea edx,[ebp-64h] 71c1c767 8d8d4cefffff lea ecx,[ebp-10B4h] 71c1c76d e8bc020000 call COMCTL32!CMarkup::_GetNextAnchorTag (71c1ca2e)

Owner: bruen...@google.com

[derekbruening]

From bruen...@google.com on December 13, 2013 21:20:20

**\ TODO solutions?

Currently we turn: A: mov eax,dword ptr [eax] B: mov dword ptr [esp],eax C: ret

Into: A: lea edx,[eax] A: mov eax,dword ptr [eax] B: mov dword ptr [edx],0 B: mov dword ptr [esp],eax C: ret

Solutions:

1) Spill a register -- but then best to move later than app2app.

2) Use push-mem to do the mem2mem?

For original code like this:

 A: mov     eax,[eax+X]
 B: push    eax
 C: ret

We transform it to:

 A: push    [eax+X]
 C: mov     dword ptr [eax+X],0
 C: ret

We can't do that for code not using a push, as we'd have to do a lea to get our push to line up -- and then we can't relocate on a fault in our inserted push, as esp is wrong. But since the cases of no push so far always have an xchg, we can do this:

 A: xchg    eax,esp
 B: mov     eax,dword ptr [eax]
 C: mov     dword ptr [esp],eax
 D: ret

=> A: pop [eax] B: mov esp,eax D: mov dword ptr [esp-4],0 D: ret

Except this leads to errors for which we'd need new exceptions:

Dr.M Error #1: UNADDRESSABLE ACCESS: writing 0x0103dde0-0x0103dde4 4 byte(s) Dr.M # 0 _chkstk Dr.M # 1 _initterm_e Dr.M # 2 KERNEL32.dll!BaseThreadInitThunk Dr.M Note: @0:00:07.940 in thread 3444 Dr.M Note: instruction: xchg %esp %eax -> %esp %eax Dr.M Dr.M Error #2: UNADDRESSABLE ACCESS: writing 0x0103dddc-0x0103dde0 4 byte(s) Dr.M # 0 _chkstk Dr.M # 1 _initterm_e Dr.M # 2 KERNEL32.dll!BaseThreadInitThunk Dr.M Note: @0:00:07.940 in thread 3444 Dr.M Note: instruction: ret %esp (%esp) -> %esp Dr.M Dr.M Error #3: UNINITIALIZED READ: reading 0x0103dde0-0x0103dde4 4 byte(s) Dr.M # 0 _chkstk Dr.M # 1 _initterm_e Dr.M # 2 KERNEL32.dll!BaseThreadInitThunk Dr.M Note: @0:00:07.940 in thread 3444 Dr.M Note: instruction: ret %esp (%esp) -> %esp

[derekbruening]

From bruen...@google.com on December 14, 2013 07:09:31

3) We don't need to write a zero on the stack, so we could just replace the 1st load with xchg, which will write a stack address (not very retaddr-looking so it should be as good as a zero) to the retaddr slot. An xchg with memory locks the bus, but that's vs 2 stores and a load if we spill a reg and write a zero; plus, it's a very simple transformation.

[derekbruening]

From derek.br...@gmail.com on December 14, 2013 08:06:44

This issue was closed by revision r1661 .

Status: Fixed