stack corruption in X86 DAG->DAG Instruction Selection

[llvmbot]

Bugzilla Link	15124
Version	3.0
OS	Windows NT
Attachments	assembly code, llvm ir
Reporter	LLVM Bugzilla Contributor
CC	@asl,@isanbard

Extended Description

I recently ran into a case, where x86 fails to run correctly.

Checking into the emitted code, it seems that the local memory anchored at ebp has conflict with stack memory anchored at esp prepared for function calls.

For example, let us say:

foo() { ... call bar(); }

There is some local data for foo() at ebp-170h. while preparing the call to bar, some data is written to esp+17h. The problem is that esp+17h and ebp-170h point to the same address, and therefore the data is corrupted.

See the attached llvm ir and assembly code. The conflict happens at line 292 and 294 in the assembly code. The address pointed by esp with offsets is the same as two of them addressed by sbp with offsets.

[isanbard]

Any update on this? :-)

[llvmbot]

Cloned to rdar://15266303

[isanbard]

Ping?

[llvmbot]

Peng, Can you bisect it and find out when (on which commit) this bug started ?

[llvmbot]

it seems that the problem is from X86 DAG->DAG Instruction Selection:

see the machine instructions after # IR Dump After X86 DAG->DAG Instruction Selection (using llc (llvm 3.0 on win32))

The first three lines and the last two lines alone together are used to compute "sin" for some double number.

• line 1: move the stack pointer down 8

• line 2: copy the updated stack pointer to a base register

• line 3: copy a double number to location pointed by the base register

• line end-1: to the last call "sin" to compute the result

• line end: move the stack pointer up 8

The problem is that there are many other instructions inserted between them, and these instructions include stack allocations. This causes:

1. "sin" function gets the wrong value to compute because the stack pointer moves and wrong value is received.

2. the function call after line end could get wrong values because after line end the stack pointer is pointing to useful data.

   ADJCALLSTACKDOWN32 8, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP ; line 1 %vreg187 = COPY %ESP; GR32:%vreg187 ; line 2 MOVSDmr %vreg187, 1, %noreg, 0, %noreg, %vreg36; mem:ST8[Stack] GR32:%vreg187 FR64:%vreg36 ; line 3 %vreg188 = MOV32rm %vreg112, 1, %noreg, 252, %noreg; mem:LD4[%108] GR32:%vreg188,%vreg112 %vreg189 = MOV32rm %vreg112, 1, %noreg, 256, %noreg; mem:LD4[%111] GR32:%vreg189,%vreg112 %vreg190 = MOVSDrm <fi#0>, 1, %noreg, 120, %noreg; mem:LD8[%85] FR64:%vreg190 %vreg191 = MOVSDrm <fi#0>, 1, %noreg, 96, %noreg; mem:LD8[%87] FR64:%vreg191 %vreg192 = MOVSDrm <fi#0>, 1, %noreg, 88, %noreg; mem:LD8[%92] FR64:%vreg192 %vreg193 = MOVSDrm <fi#0>, 1, %noreg, 24, %noreg; mem:LD8[%89] FR64:%vreg193 %vreg194 = MOVSDrm <fi#0>, 1, %noreg, 80, %noreg; mem:LD8[%94] FR64:%vreg194 %vreg195 = MOV32ri 8; GR32:%vreg195 %EAX = COPY %vreg195; GR32:%vreg195 WIN_ALLOCA %EAX<imp-def,dead>, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP %vreg196 = COPY %ESP; GR32:%vreg196 MOV32mr %vreg196, 1, %noreg, 0, %noreg, %vreg16; mem:ST4[%114] GR32:%vreg196,%vreg16 %vreg197 = MOV32ri 72; GR32:%vreg197 %EAX = COPY %vreg197; GR32:%vreg197 WIN_ALLOCA %EAX<imp-def,dead>, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP %vreg198 = COPY %ESP; GR32:%vreg198 MOVSDmr %vreg198, 1, %noreg, 56, %noreg, %vreg194; mem:ST8[%116+56] GR32:%vreg198 FR64:%vreg194 MOVSDmr %vreg198, 1, %noreg, 64, %noreg, %vreg17; mem:ST8[%116+64] GR32:%vreg198 FR64:%vreg17 MOVSDmr %vreg198, 1, %noreg, 48, %noreg, %vreg192; mem:ST8[%116+48] GR32:%vreg198 FR64:%vreg192 MOVSDmr %vreg198, 1, %noreg, 32, %noreg, %vreg191; mem:ST8[%116+32] GR32:%vreg198 FR64:%vreg191 MOVSDmr %vreg198, 1, %noreg, 24, %noreg, %vreg36; mem:ST8[%116+24] GR32:%vreg198 FR64:%vreg36 MOVSDmr %vreg198, 1, %noreg, 16, %noreg, %vreg190; mem:ST8[%116+16] GR32:%vreg198 FR64:%vreg190 MOVSDmr %vreg198, 1, %noreg, 8, %noreg, %vreg190; mem:ST8[%116+8] GR32:%vreg198 FR64:%vreg190 MOVSDmr %vreg198, 1, %noreg, 0, %noreg, %vreg190; mem:ST8[%116] GR32:%vreg198 FR64:%vreg190 %vreg199 = MOVSDrm %noreg, 1, %noreg, <cp#4>, %noreg; mem:LD8ConstantPool FR64:%vreg199 %vreg200 = FsXORPDrr %vreg193, %vreg199; FR64:%vreg200,%vreg193,%vreg199 MOVSDmr %vreg198, 1, %noreg, 40, %noreg, %vreg200; mem:ST8[%116+40] GR32:%vreg198 FR64:%vreg200 %vreg201 = MOV32ri 16; GR32:%vreg201 %EAX = COPY %vreg201; GR32:%vreg201 WIN_ALLOCA %EAX<imp-def,dead>, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP %vreg202 = COPY %ESP; GR32:%vreg202 MOV32mr %vreg202, 1, %noreg, 8, %noreg, %vreg19; mem:ST4[%118+8] GR32:%vreg202,%vreg19 MOV32mr %vreg202, 1, %noreg, 4, %noreg, %vreg18; mem:ST4[%118+4] GR32:%vreg202,%vreg18 MOV32mr %vreg202, 1, %noreg, 0, %noreg, %vreg0; mem:ST4[%118] GR32:%vreg202,%vreg0 CALLpcrel32 , %EAX<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP, ... ; line end-1 ADJCALLSTACKUP32 8, 0, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP ; line end

[llvmbot]

A correction:

After reading the assembly code, the problem is when stack memory allocation is happening between the stack subtraction and the actual function call.

For example, the correct call to sin is:

sub esp, 8 move [esp], xmm0 call sin add esp, 8

While the generated code has:

sub esp, 8 move [esp], xmm0 move eax, 10h call chkstk ; allocate more stack mov esi,esp
mov eax,dword ptr [ebp-278h]
mov dword ptr [esi+8],eax
mov eax,dword ptr [ebp-274h]
mov dword ptr [esi+4],eax
mov eax,dword ptr [ebp-26Ch]
mov dword ptr [esi],eax
call sin add esp, 8

This causes the stack pointer is pointing to the some user data, i.e., 8 bytes above the true stack top.