JITting of x32 code on x64 fails with crash or instruction selection error.

[Quuxplusone]

Bugzilla Link	PR34268
Status	NEW
Importance	P normal
Reported by	AlexDenisov (1101.debian@gmail.com)
Reported on	2017-08-21 14:18:50 -0700
Last modified on	2020-11-29 13:30:38 -0800
Version	4.0
Hardware	Macintosh Linux
CC	1101.debian@gmail.com, dblaikie@gmail.com, glaubitz@physik.fu-berlin.de, harald@gigawatt.nl, lhames@gmail.com, llvm-bugs@lists.llvm.org, llvm-dev@ndave.org, llvm-dev@redking.me.uk, stefan.graenitz@gmail.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also	PR36743

I compile a simple code:

// main.c
#include <stdio.h>

int square(int x) {
  return x * x;
}

int main() {
  printf("%d\n", square(4));
  return 0;
}

Using the following command:

> clang-4.0 main.c -c -emit-llvm -m32

Then I try to run the bitcode using lli. Based on jit kind it either crashes or
shows an instruction selection error:

> lli-4.0 -jit-kind=orc-mcjit main.bc
0  libLLVM-4.0.so.1 0x00007fa69adc49a8
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 56
1  libLLVM-4.0.so.1 0x00007fa69adc2b2e llvm::sys::RunSignalHandlers() + 62
2  libLLVM-4.0.so.1 0x00007fa69adc2c7c
3  libpthread.so.0  0x00007fa69a4aa390
4  libpthread.so.0  0x00007fa69d81e026
Stack dump:
0.  Program arguments: lli-4.0 -jit-kind=orc-mcjit main.bc
fish: “lli-4.0 -jit-kind=orc-mcjit mai…” terminated by signal SIGSEGV (Address
boundary error)

> lli-4.0  main.bc
0  libLLVM-4.0.so.1 0x00007f2377ae19a8
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 56
1  libLLVM-4.0.so.1 0x00007f2377adfb2e llvm::sys::RunSignalHandlers() + 62
2  libLLVM-4.0.so.1 0x00007f2377adfc7c
3  libpthread.so.0  0x00007f23771c7390
4  libpthread.so.0  0x00007f237a53b026
Stack dump:
0.  Program arguments: lli-4.0 main.bc
fish: “lli-4.0  main.bc” terminated by signal SIGSEGV (Address boundary error)

> lli-4.0 -jit-kind=orc-lazy main.bc
LLVM ERROR: Cannot select: 0x199d480: ch,glue = X86ISD::CALL 0x199d418,
0x199d758, Register:i32 %EDI, Register:i32 %ESI, Register:i8 %AL,
RegisterMask:Untyped, 0x199d418:1
  0x199d758: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<i32 (i8*, ...)* @printf> 0
    0x199d140: i32 = TargetGlobalAddress<i32 (i8*, ...)* @printf> 0
  0x199ce00: i32 = Register %EDI
  0x199d2e0: i32 = Register %ESI
  0x199d3b0: i8 = Register %AL
  0x199ced0: Untyped = RegisterMask
  0x199d418: ch,glue = CopyToReg 0x199d348, Register:i8 %AL, Constant:i8<0>, 0x199d348:1
    0x199d3b0: i8 = Register %AL
    0x199d210: i8 = Constant<0>
    0x199d348: ch,glue = CopyToReg 0x199d278, Register:i32 %ESI, 0x199d070, 0x199d278:1
      0x199d2e0: i32 = Register %ESI
      0x199d070: i32,ch,glue = CopyFromReg 0x199cfa0, Register:i32 %EAX, 0x199cfa0:1
        0x199d008: i32 = Register %EAX
        0x199cfa0: ch,glue = callseq_end 0x199cf38, TargetConstant:i32<0>, TargetConstant:i32<0>, 0x199cf38:1
          0x199cd30: i32 = TargetConstant<0>
          0x199cd30: i32 = TargetConstant<0>
          0x199cf38: ch,glue = X86ISD::CALL 0x199ce68, 0x199d7c0, Register:i32 %EDI, RegisterMask:Untyped, 0x199ce68:1
            0x199d7c0: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<i32 (i32)* @square> 0
              0x199d0d8: i32 = TargetGlobalAddress<i32 (i32)* @square> 0
            0x199ce00: i32 = Register %EDI
            0x199ced0: Untyped = RegisterMask
            0x199ce68: ch,glue = CopyToReg 0x199cd98, Register:i32 %EDI, Constant:i32<4>
              0x199ce00: i32 = Register %EDI
              0x199ccc8: i32 = Constant<4>
      0x199d278: ch,glue = CopyToReg 0x199d1a8, Register:i32 %EDI, 0x199d6f0
        0x199ce00: i32 = Register %EDI
        0x199d6f0: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<[4 x i8]* @"$static.0"> 0
          0x199d688: i32 = TargetGlobalAddress<[4 x i8]* @"$static.0"> 0
In function: main

===

I tested it only on Linux, but other systems might be affected as well.

[Quuxplusone]

CC'ing Lang Hames, jfyi.

[Quuxplusone]

Looks like this may be the same bug as http://llvm.org/PR36743. If so, it is not JIT specific, but even if it were fixed it is possible we would need to do some work to support the X32 ABI in ORC.

[Quuxplusone]

Recommended workaround: build LLVM and Orc for x32:

    cmake -DLLVM_BUILD_32_BITS=On path-to-llvm-sources

[Quuxplusone]

I think this needs some clarification. PR36743 states that: "-fno-plt -mx32" at
any optimization level higher than -O0 will break at isel
In this report neither -fno-plt nor -Ox are involved. It's about -m32 and not -
mx32. While I don't know much about their difference, I see:

$ bin/clang --version
clang version 9.0.0 (https://github.com/llvm/llvm-project.git
51dc59d0903fc86b37e9d910aa6f33dd87fdaae5)
Target: x86_64-apple-darwin18.5.0
Thread model: posix

$ cat main.c
int square(int x) { return x * x; }
int main(int argc, char **argv) { return square(argc); }

$ bin/clang -m32 main.c -o main-m32
$ ./main-m32
$ echo $?
1

$ bin/clang -m32 -mx32 main.c -o main-m32
$ ./main-m32
$ echo $?
1

$ bin/clang -c -emit-llvm -m32 main.c -o - | bin/lli
Illegal instruction: 4

$ bin/clang -c -emit-llvm -m32 -mx32 main.c -o - | bin/lli
$ echo $?
1

This means passing -m32 and avoiding the -mx32 flag causes clang to emit
bitcode that lli doesn't interpret correctly.
How the failure manifests depends on the JIT engine:

$ bin/clang -c -emit-llvm -m32 main.c -o - | bin/lli -jit-kind=orc-lazy
Segmentation fault: 11
$ bin/clang -c -emit-llvm -m32 main.c -o - | bin/lli -jit-kind=orc-mcjit
Illegal instruction: 4

In fact I can reproduce the instruction selection issue (only) with an
incompatible target (I couldn't link that statically either):

$ bin/clang -c -emit-llvm -mx32 --target=i386-apple-darwin main.c -o - | bin/lli
$ echo $?
1

$ bin/clang -c -emit-llvm -mx32 --target=i386-unknown-unknown main.c -o - |
bin/lli
LLVM ERROR: Cannot select: t9: ch,glue = X86ISD::CALL t7, t17, Register:i32
$edi, RegisterMask:Untyped, t7:1
  t17: i32 = X86ISD::Wrapper TargetGlobalAddress:i32<i32 (i32)* @square> 0
    t16: i32 = TargetGlobalAddress<i32 (i32)* @square> 0
  t6: i32 = Register $edi
  t8: Untyped = RegisterMask
  t7: ch,glue = CopyToReg t5, Register:i32 $edi, t3
    t6: i32 = Register $edi
    t3: i32,ch = CopyFromReg t0, Register:i32 %5
      t2: i32 = Register %5
In function: main

So either I am missing something, or the situation changed entirely since the
report, or there was some other factor that we didn't consider.