stack corruption in X86 DAG->DAG Instruction Selection

[Quuxplusone]

Bugzilla Link	PR15124
Status	NEW
Importance	P release blocker
Reported by	Peng Cheng (gm4cheng@gmail.com)
Reported on	2013-01-30 14:35:37 -0800
Last modified on	2013-12-29 01:05:16 -0800
Version	3.0
Hardware	PC Windows NT
CC	anton@korobeynikov.info, baldrick@free.fr, gm4cheng@gmail.com, isanbard@gmail.com, llvm-bugs@lists.llvm.org, nadav.rotem@me.com, rafael@espindo.la
Fixed by commit(s)
Attachments	win32iremit.s (15777 bytes, text/plain) win32iremit_report.txt (20339 bytes, text/plain)
Blocks
Blocked by
See also

Created attachment 9950
assembly code

I recently ran into a case, where x86 fails to run correctly.

Checking into the emitted code, it seems that the local memory anchored at ebp
has conflict with stack memory anchored at esp prepared for function calls.

For example, let us say:

foo()
{
  ...
  call bar();
}

There is some local data for foo() at ebp-170h.  while preparing the call to
bar, some data is written to esp+17h.  The problem is that esp+17h and ebp-170h
point to the same address, and therefore the data is corrupted.

See the attached llvm ir and assembly code.  The conflict happens at line 292
and 294 in the assembly code.  The address pointed by esp with offsets is the
same as two of them addressed by sbp with offsets.

[Quuxplusone]

Attached win32iremit.s (15777 bytes, text/plain): assembly code

[Quuxplusone]

Attached win32iremit_report.txt (20339 bytes, text/plain): llvm ir

[Quuxplusone]

A correction:

After reading the assembly code, the problem is when stack memory allocation is
happening between the stack subtraction and the actual function call.

For example, the correct call to sin is:

sub esp, 8
move [esp], xmm0
call sin
add esp, 8

While the generated code has:

sub esp, 8
move [esp], xmm0
move eax, 10h
call chkstk ; allocate more stack
mov         esi,esp
mov         eax,dword ptr [ebp-278h]
mov         dword ptr [esi+8],eax
mov         eax,dword ptr [ebp-274h]
mov         dword ptr [esi+4],eax
mov         eax,dword ptr [ebp-26Ch]
mov         dword ptr [esi],eax
call sin
add esp, 8

This causes the stack pointer is pointing to the some user data, i.e., 8 bytes
above the true stack top.

[Quuxplusone]

it seems that the problem is from X86 DAG->DAG Instruction Selection:

see the machine instructions after # *** IR Dump After X86 DAG->DAG Instruction
Selection *** (using llc (llvm 3.0 on win32))

The first three lines and the last two lines alone together are used to compute
"sin" for some double number.

- line 1: move the stack pointer down 8
- line 2: copy the updated stack pointer to a base register
- line 3: copy a double number to location pointed by the base register

- line end-1: to the last call "sin" to compute the result
- line end: move the stack pointer up 8

The problem is that there are many other instructions inserted between them,
and these instructions include stack allocations.  This causes:

1. "sin" function gets the wrong value to compute because the stack pointer
moves and wrong value is received.
2. the function call after line end could get wrong values because after line
end the stack pointer is pointing to useful data.

    ADJCALLSTACKDOWN32 8, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP<imp-use>
; line 1
    %vreg187<def> = COPY %ESP; GR32:%vreg187
; line 2
    MOVSDmr %vreg187, 1, %noreg, 0, %noreg, %vreg36; mem:ST8[Stack] GR32:%vreg187
FR64:%vreg36               ; line 3
    %vreg188<def> = MOV32rm %vreg112, 1, %noreg, 252, %noreg; mem:LD4[%108]
GR32:%vreg188,%vreg112
    %vreg189<def> = MOV32rm %vreg112, 1, %noreg, 256, %noreg; mem:LD4[%111]
GR32:%vreg189,%vreg112
    %vreg190<def> = MOVSDrm <fi#0>, 1, %noreg, 120, %noreg; mem:LD8[%85]
FR64:%vreg190
    %vreg191<def> = MOVSDrm <fi#0>, 1, %noreg, 96, %noreg; mem:LD8[%87]
FR64:%vreg191
    %vreg192<def> = MOVSDrm <fi#0>, 1, %noreg, 88, %noreg; mem:LD8[%92]
FR64:%vreg192
    %vreg193<def> = MOVSDrm <fi#0>, 1, %noreg, 24, %noreg; mem:LD8[%89]
FR64:%vreg193
    %vreg194<def> = MOVSDrm <fi#0>, 1, %noreg, 80, %noreg; mem:LD8[%94]
FR64:%vreg194
    %vreg195<def> = MOV32ri 8; GR32:%vreg195
    %EAX<def> = COPY %vreg195; GR32:%vreg195
    WIN_ALLOCA %EAX<imp-def,dead>, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>,
%ESP<imp-use>
    %vreg196<def> = COPY %ESP; GR32:%vreg196
    MOV32mr %vreg196, 1, %noreg, 0, %noreg, %vreg16; mem:ST4[%114]
GR32:%vreg196,%vreg16
    %vreg197<def> = MOV32ri 72; GR32:%vreg197
    %EAX<def> = COPY %vreg197; GR32:%vreg197
    WIN_ALLOCA %EAX<imp-def,dead>, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>,
%ESP<imp-use>
    %vreg198<def> = COPY %ESP; GR32:%vreg198
    MOVSDmr %vreg198, 1, %noreg, 56, %noreg, %vreg194; mem:ST8[%116+56]
GR32:%vreg198 FR64:%vreg194
    MOVSDmr %vreg198, 1, %noreg, 64, %noreg, %vreg17; mem:ST8[%116+64]
GR32:%vreg198 FR64:%vreg17
    MOVSDmr %vreg198, 1, %noreg, 48, %noreg, %vreg192; mem:ST8[%116+48]
GR32:%vreg198 FR64:%vreg192
    MOVSDmr %vreg198, 1, %noreg, 32, %noreg, %vreg191; mem:ST8[%116+32]
GR32:%vreg198 FR64:%vreg191
    MOVSDmr %vreg198, 1, %noreg, 24, %noreg, %vreg36; mem:ST8[%116+24]
GR32:%vreg198 FR64:%vreg36
    MOVSDmr %vreg198, 1, %noreg, 16, %noreg, %vreg190; mem:ST8[%116+16]
GR32:%vreg198 FR64:%vreg190
    MOVSDmr %vreg198, 1, %noreg, 8, %noreg, %vreg190; mem:ST8[%116+8]
GR32:%vreg198 FR64:%vreg190
    MOVSDmr %vreg198, 1, %noreg, 0, %noreg, %vreg190; mem:ST8[%116] GR32:%vreg198
FR64:%vreg190
    %vreg199<def> = MOVSDrm %noreg, 1, %noreg, <cp#4>, %noreg;
mem:LD8[ConstantPool](align=16) FR64:%vreg199
    %vreg200<def> = FsXORPDrr %vreg193, %vreg199; FR64:%vreg200,%vreg193,%vreg199
    MOVSDmr %vreg198, 1, %noreg, 40, %noreg, %vreg200; mem:ST8[%116+40]
GR32:%vreg198 FR64:%vreg200
    %vreg201<def> = MOV32ri 16; GR32:%vreg201
    %EAX<def> = COPY %vreg201; GR32:%vreg201
    WIN_ALLOCA %EAX<imp-def,dead>, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>,
%ESP<imp-use>
    %vreg202<def> = COPY %ESP; GR32:%vreg202
    MOV32mr %vreg202, 1, %noreg, 8, %noreg, %vreg19; mem:ST4[%118+8]
GR32:%vreg202,%vreg19
    MOV32mr %vreg202, 1, %noreg, 4, %noreg, %vreg18; mem:ST4[%118+4]
GR32:%vreg202,%vreg18
    MOV32mr %vreg202, 1, %noreg, 0, %noreg, %vreg0; mem:ST4[%118]
GR32:%vreg202,%vreg0
    CALLpcrel32 <es:sin>, %EAX<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP<imp-
use>, ...                                         ; line end-1
    ADJCALLSTACKUP32 8, 0, %ESP<imp-def,dead>, %EFLAGS<imp-def,dead>, %ESP<imp-
use>                                       ; line end

[Quuxplusone]

Peng, Can you bisect it and find out when (on which commit) this bug started ?

[Quuxplusone]

Ping?

[Quuxplusone]

Cloned to rdar://15266303

[Quuxplusone]

Any update on this? :-)