X86: tail call to function with two i256 arguments miscompiled, clobbers stack

[dwightguth]

This bug can be reproduced on the main branch of llvm on the X86 target.

Program that was miscompiled:

define tailcc ptr @f2(ptr %0, ptr %1, ptr %2, ptr %3, i256 %4, i256 %5, ptr %6) {
  ret ptr %0
}

define tailcc ptr @f1(ptr %0) {
  %ret = musttail call tailcc ptr @f2(ptr %0, ptr %0, ptr %0, ptr %0, i256 0, i256 0, ptr %0)
  ret ptr %ret
}

define i32 @main() {
  %ret = call tailcc ptr @f1(ptr null)
  ret i32 0
}

To reproduce:

llc bug.ll -filetype=obj
clang bug.o
./a.out

Prior to commit fa1b6e6b34eb6382c451f3a06a7c52d7ac6ada1d ([X86]: Fix i128 argument passing under SysV ABI), this program worked correctly. It fails thereafter. I'm not entirely sure what the mechanism of the failure is. However, it's worth noting that the function call that triggers the bug is a tailcc function call and thus not subject to the SysV ABI; it's possible that the fix could be as simple as just reverting the change to the calling convention when not subject to the ABI. I'd be happy to submit such a PR if someone gives the okay.

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: Dwight Guth (dwightguth)

This bug can be reproduced on the main branch of llvm on the X86 target. Program that was miscompiled: ``` define tailcc ptr @f2(ptr %0, ptr %1, ptr %2, ptr %3, i256 %4, i256 %5, ptr %6) { ret ptr %0 } define tailcc ptr @f1(ptr %0) { %ret = musttail call tailcc ptr @f2(ptr %0, ptr %0, ptr %0, ptr %0, i256 0, i256 0, ptr %0) ret ptr %ret } define i32 @main() { %ret = call tailcc ptr @f1(ptr null) ret i32 0 } ``` To reproduce: ``` llc bug.ll -filetype=obj clang bug.o ./a.out ``` Prior to commit fa1b6e6b34eb6382c451f3a06a7c52d7ac6ada1d ([X86]: Fix i128 argument passing under SysV ABI), this program worked correctly. It fails thereafter. I'm not entirely sure what the mechanism of the failure is. However, it's worth noting that the function call that triggers the bug is a tailcc function call and thus not subject to the SysV ABI; it's possible that the fix could be as simple as just reverting the change to the calling convention when not subject to the ABI. I'd be happy to submit such a PR if someone gives the okay.