disagreement with GCC about ABI of libgcc functions on hard-float ARM

[Quuxplusone]

Bugzilla Link	PR27455
Status	NEW
Importance	P normal
Reported by	Brian Silverman (brian@peloton-tech.com)
Reported on	2016-04-21 16:39:29 -0700
Last modified on	2016-11-10 17:52:12 -0800
Version	unspecified
Hardware	Other Linux
CC	anton@korobeynikov.info, bruel38+llvm@gmail.com, compnerd@compnerd.org, dushistov@gmail.com, jistone@redhat.com, jroelofs@jroelofs.com, kristof.beyls@arm.com, llvm-bugs@lists.llvm.org, tzuhsiang.chien@gmail.com
Fixed by commit(s)
Attachments	test_complex.c (332 bytes, text/x-csrc)
Blocks
Blocked by
See also

Created attachment 16246
A simple program which demonstrates the affects

Clang and GCC disagree about the calling convention for the helper functions in
libgcc/compiler-rt on ARM with hardware floating point. Clang passes the
floating point arguments in integer registers (like with -mfloat-abi=soft), but
GCC calls them like other functions with the hard-float ABI. The only reference
to this I can find online is an LLVM development mailing list post [1]
discussing why Clang switched to always using soft-float. At least one of the
Clang developers thought GCC did this too, but it doesn't seem to currently.
Complex arithmetic is implemented with these functions even when hardware
floating point is supported, which means complex arithmetic is pretty
completely broken on platforms like this when using libgcc.

[1] http://lists.llvm.org/pipermail/llvm-dev/2014-October/077898.html

I found the code in clang's lib/CodeGen/TargetInfo.cpp which sets BuiltinCC to
CallingConv::ARM_AAPCS and the __attribute__((pcs("aapcs"))) in compiler-rt's
lib/builtins/int_lib.h which compiles the relevant functions that way. I can't
find any evidence of GCC doing either of those.

In all three copies of libgcc.a I've disassembled (Debian Jessie armhf and both
of the ones from gcc-linaro-4.9-2015.05-x86_64_arm-linux-gnueabihf) __divdc3
(for example) definitely uses the hard-float ABI. In clang+llvm-3.8.0-armv7a-
linux-gnueabihf/lib/clang/3.8.0/lib/linux/libclang_rt.builtins-armhf.a
(downloaded from llvm.org), __divdc3 is definitely using the soft-float ABI.

Attached is a simple program which demonstrates the effects.
I've tried with Clang 3.7.1 I cross-compiled for armhf myself and 3.8.0
binaries downloaded from llvm.org.
$ gcc -lm test_complex.c -O3 && ./a.out
works
$ clang -lm test_complex.c -O3 && ./a.out
broken

On 4/21/16 2:36 PM, Renato Golin via cfe-dev wrote:
> I checked with the GCC folks, and the behaviour is:
>
>  * arm-linux-gnueabi has *only* soft-float
>  * arm-linux-gnueagihf has *only* hard float
>  * arm-none-eabi has *all* variations (above) with the default being soft-
float

[Quuxplusone]

Attached test_complex.c (332 bytes, text/x-csrc): A simple program which demonstrates the affects

[Quuxplusone]

Brian,

It might happen that gcc changed its behavior silently (thus breaking the ABI). I believe that back in 2014 when this issue was first seen libgcc certainly used AAPCS calling convention.

clang change was added to effectively fix the opposite problem - we used to emit calls using AAPCS-FP calling while both libgcc and compiler-rt had only AAPCS versions (I believe I double checked this behavior at that time).

So, I'm not sure how to proceed. We cannot simply revert the change, because this will surely break ABI and backward compatibility.

[Quuxplusone]

This is certainly interesting.  Ive looked into this previously, but it is
possible that I missed something or analyzed something incorrectly.

My understanding is that the hard float/soft float choice does not impact
libgcc at all.  GCC will *generate* calls to the library using ARM_AAPCSCC.
However, it may be possible to get libgcc to be compiled ARM_AAPCS_VFPCC.  In
this case you end up with the calling convention mismatch, and that will
certainly fail.

In the case of "soft" and "soft fp", the calling convention matches and
everything is happy.  In the case of "hard", most of the calls can be lowered
directly to instructions rather than a library call.  In such a scenario, again
we are safe.  What remains is the last few cases where we generate library
calls.

Retaining the current behavior of compiling compiler-rt as ARM_AAPCSCC means
that we remain compatible with gcc's code generation irrespective of the
floating point ABI used.

This area is certainly under documented, but AIUI, the ABI for the libgcc calls
for AAPCSCC rather than AAPCS_VFPCC.  If there is documentation indicating
otherwise (or worst case scenario, code) or perhaps things have changed in the
past year or so, we can certainly adjust the behavior of clang/llvm.

If GCC does use both calling conventions when making library calls, we can
always do two builds with differing calling conventions and link them
appropriately.  If necessary, we could add thunks to adjust for the calling
convention as well.

[Quuxplusone]

(In reply to comment #2)
> If GCC does use both calling conventions when making library calls, we can
> always do two builds with differing calling conventions and link them
> appropriately.  If necessary, we could add thunks to adjust for the calling
> convention as well.
One of the problems is that there is no mangling which would allow us to
distinguish between different calling conventions....

[Quuxplusone]

Clang is currently not compatible with GCC's code generation. I do have code indicating that GCC uses AAPCS_VFPCC for functions (calls and definitions) in libgcc. The Debian Jessie armhf libgcc.a is an example of code, and the compiler from the same place generates matching calls. The gcc-linaro-4.9-2015.05-x86_64_arm-linux-gnueabihf (amd64 to armhf cross) toolchain has both of those too, and the libgcc_s shared objects from both places also have hard-float code.

I'm not sure changing this would break much. From what I'm seeing, these functions get statically linked into all binaries which use them. This means that (unless somebody's doing something pretty unusual, like somehow using the code from libgcc_s.so or the compiler-rt equivalent) changing compiler-rt and clang together won't break anybody, as long as they're using matched versions.

It sounds like Clang needs two versions of the lib and some kind of option for how to generate calls to be fully compatible with GCC.

I'm currently working on compiling a libclang from compiler-rt to try using at compile time. I think by linking that into the binary first I'll get the soft-fp versions of the functions, and it'll work. That's definitely a somewhat fragile workaround though.