search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.03k stars 11.57k forks source link

Miscompilation when linking an LTO version of newlib for ARM #44869

Open llvmbot opened 4 years ago

llvmbot commented 4 years ago
Bugzilla Link 45524
Version unspecified
OS Linux
Reporter LLVM Bugzilla Contributor
CC @asl,@john-brawn-arm

Extended Description

Hi,

I'm working on an embedded ARM project using LLVM to do cross compilation, in which we need to do whole-program transformations on both the application and libraries (statically linked newlib and compiler-rt). We therefore chose to compile and link everything with LTO. However, miscompilation happens where a printf("\n") in C is compiled into a call to @​llvm.trap() and an unreachable instruction in IR when we link against an LTO version of newlib.

After some debugging, we found out that the problem is caused by mismatched calling convention. Specifically,

  1. In the compiling stage, LibCallSimplifier replaces the original call to printf() with ARM AAPCS calling convention by a call to putchar() with LLVM's default calling convention.

  2. In the function importing stage of LTO, the function body of putchar() with ARM AAPCS calling convention is imported from the LTO newlib into the single module.

  3. When running InstCombine pass in LTO, due to mismatched calling convention (default in the CallInst vs. ARM AAPCS in putchar()'s definition), the CallInst is transformed into an unreachable instruction (An explanation can be found in https://llvm.org/docs/FAQ.html#why-does-instcombine-simplifycfg-turn-a-call-to-a-function-with-a-mismatched-calling-convention-into-unreachable-why-not-make-the-verifier-reject-it).

I don't know if LLVM should support compiling low-level infrastructure (such as C library) with LTO, but my speculation is that LibCallSimplifier assumes the C library to be in native code which makes it disregard the original call's calling convention.

Another interesting fact is that this is not only the case with LTO, but also with ThinLTO. Specifically, I tried the following combinations, some of which are actually working:

application:LTO + newlib:LTO -> unreachable application:ThinLTO + newlib:ThinLTO -> unreachable application:LTO + newlib:ThinLTO -> working application:ThinLTO + newlib:LTO -> working

For now I'm using one of the last two working configurations for my project, but it would be great to see all of the above working.

john-brawn-arm commented 3 years ago

Patch which fixes this by keeping the calling convention of the call that's being replaced: https://reviews.llvm.org/D89400.