JITting of x32 code on x64 fails with crash or instruction selection error.

[AlexDenisov]

Bugzilla Link	34268
Version	4.0
OS	Linux
CC	@AlexDenisov,@dwblaikie,@glaubitz,@hvdijk,@lhames,@RKSimon,@weliveindetail

Extended Description

I compile a simple code:

// main.c

include

int square(int x) { return x * x; }

int main() { printf("%d\n", square(4)); return 0; }

Using the following command:

clang-4.0 main.c -c -emit-llvm -m32

Then I try to run the bitcode using lli. Based on jit kind it either crashes or shows an instruction selection error:

lli-4.0 -jit-kind=orc-mcjit main.bc 0 libLLVM-4.0.so.1 0x00007fa69adc49a8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 56 1 libLLVM-4.0.so.1 0x00007fa69adc2b2e llvm::sys::RunSignalHandlers() + 62 2 libLLVM-4.0.so.1 0x00007fa69adc2c7c 3 libpthread.so.0 0x00007fa69a4aa390 4 libpthread.so.0 0x00007fa69d81e026 Stack dump:

0. Program arguments: lli-4.0 -jit-kind=orc-mcjit main.bc fish: “lli-4.0 -jit-kind=orc-mcjit mai…” terminated by signal SIGSEGV (Address boundary error)

lli-4.0 main.bc 0 libLLVM-4.0.so.1 0x00007f2377ae19a8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 56 1 libLLVM-4.0.so.1 0x00007f2377adfb2e llvm::sys::RunSignalHandlers() + 62 2 libLLVM-4.0.so.1 0x00007f2377adfc7c 3 libpthread.so.0 0x00007f23771c7390 4 libpthread.so.0 0x00007f237a53b026 Stack dump:

0. Program arguments: lli-4.0 main.bc fish: “lli-4.0 main.bc” terminated by signal SIGSEGV (Address boundary error)

lli-4.0 -jit-kind=orc-lazy main.bc LLVM ERROR: Cannot select: 0x199d480: ch,glue = X86ISD::CALL 0x199d418, 0x199d758, Register:i32 %EDI, Register:i32 %ESI, Register:i8 %AL, RegisterMask:Untyped, 0x199d418:1 0x199d758: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<i32 (i8, ...) @​printf> 0 0x199d140: i32 = TargetGlobalAddress<i32 (i8, ...) @​printf> 0 0x199ce00: i32 = Register %EDI 0x199d2e0: i32 = Register %ESI 0x199d3b0: i8 = Register %AL 0x199ced0: Untyped = RegisterMask 0x199d418: ch,glue = CopyToReg 0x199d348, Register:i8 %AL, Constant:i8<0>, 0x199d348:1 0x199d3b0: i8 = Register %AL 0x199d210: i8 = Constant<0> 0x199d348: ch,glue = CopyToReg 0x199d278, Register:i32 %ESI, 0x199d070, 0x199d278:1 0x199d2e0: i32 = Register %ESI 0x199d070: i32,ch,glue = CopyFromReg 0x199cfa0, Register:i32 %EAX, 0x199cfa0:1 0x199d008: i32 = Register %EAX 0x199cfa0: ch,glue = callseq_end 0x199cf38, TargetConstant:i32<0>, TargetConstant:i32<0>, 0x199cf38:1 0x199cd30: i32 = TargetConstant<0> 0x199cd30: i32 = TargetConstant<0> 0x199cf38: ch,glue = X86ISD::CALL 0x199ce68, 0x199d7c0, Register:i32 %EDI, RegisterMask:Untyped, 0x199ce68:1 0x199d7c0: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<i32 (i32) @​square> 0 0x199d0d8: i32 = TargetGlobalAddress<i32 (i32) @​square> 0 0x199ce00: i32 = Register %EDI 0x199ced0: Untyped = RegisterMask 0x199ce68: ch,glue = CopyToReg 0x199cd98, Register:i32 %EDI, Constant:i32<4> 0x199ce00: i32 = Register %EDI 0x199ccc8: i32 = Constant<4> 0x199d278: ch,glue = CopyToReg 0x199d1a8, Register:i32 %EDI, 0x199d6f0 0x199ce00: i32 = Register %EDI 0x199d6f0: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<[4 x i8] @"$static.0"> 0 0x199d688: i32 = TargetGlobalAddress<[4 x i8] @"$static.0"> 0 In function: main

===

I tested it only on Linux, but other systems might be affected as well.

[llvmbot]

mentioned in issue llvm/llvm-bugzilla-archive#36743

[weliveindetail]

I think this needs some clarification. llvm/llvm-bugzilla-archive#36743 states that: "-fno-plt -mx32" at any optimization level higher than -O0 will break at isel In this report neither -fno-plt nor -Ox are involved. It's about -m32 and not -mx32. While I don't know much about their difference, I see:

$ bin/clang --version clang version 9.0.0 (https://github.com/llvm/llvm-project.git 51dc59d0903fc86b37e9d910aa6f33dd87fdaae5) Target: x86_64-apple-darwin18.5.0 Thread model: posix

$ cat main.c int square(int x) { return x * x; } int main(int argc, char **argv) { return square(argc); }

$ bin/clang -m32 main.c -o main-m32 $ ./main-m32 $ echo $? 1

$ bin/clang -m32 -mx32 main.c -o main-m32 $ ./main-m32 $ echo $? 1

$ bin/clang -c -emit-llvm -m32 main.c -o - | bin/lli Illegal instruction: 4

$ bin/clang -c -emit-llvm -m32 -mx32 main.c -o - | bin/lli $ echo $? 1

This means passing -m32 and avoiding the -mx32 flag causes clang to emit bitcode that lli doesn't interpret correctly. How the failure manifests depends on the JIT engine:

$ bin/clang -c -emit-llvm -m32 main.c -o - | bin/lli -jit-kind=orc-lazy Segmentation fault: 11 $ bin/clang -c -emit-llvm -m32 main.c -o - | bin/lli -jit-kind=orc-mcjit Illegal instruction: 4

In fact I can reproduce the instruction selection issue (only) with an incompatible target (I couldn't link that statically either):

$ bin/clang -c -emit-llvm -mx32 --target=i386-apple-darwin main.c -o - | bin/lli $ echo $? 1

$ bin/clang -c -emit-llvm -mx32 --target=i386-unknown-unknown main.c -o - | bin/lli LLVM ERROR: Cannot select: t9: ch,glue = X86ISD::CALL t7, t17, Register:i32 $edi, RegisterMask:Untyped, t7:1 t17: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<i32 (i32) @​square> 0 t16: i32 = TargetGlobalAddress<i32 (i32) @​square> 0 t6: i32 = Register $edi t8: Untyped = RegisterMask t7: ch,glue = CopyToReg t5, Register:i32 $edi, t3 t6: i32 = Register $edi t3: i32,ch = CopyFromReg t0, Register:i32 %5 t2: i32 = Register %5 In function: main

So either I am missing something, or the situation changed entirely since the report, or there was some other factor that we didn't consider.

[AlexDenisov]

Recommended workaround: build LLVM and Orc for x32:

cmake -DLLVM_BUILD_32_BITS=On path-to-llvm-sources

[lhames]

Looks like this may be the same bug as llvm/llvm-bugzilla-archive#36743 . If so, it is not JIT specific, but even if it were fixed it is possible we would need to do some work to support the X32 ABI in ORC.

[AlexDenisov]

CC'ing Lang Hames, jfyi.