calling convention mismatch CoreRT/clang on ARM

[iarischenko]

The following code leads to SIGSEGV:

using System;

namespace test
{
    class Program
    {
        static void Main(string[] args)
        {
            var num = Math.Sqrt(8);
            Console.WriteLine("Hello World! double: " + num);
        }
    }
}

Thread 1 "test" received signal SIGSEGV, Segmentation fault.
_IO_vsnprintf (string=0x21 <error: Cannot access memory at address 0x21>, maxlen=<optimized out>, format=format@entry=0x1a <error: Cannot access memory at address 0x1a>, args=..., 
    args@entry=...) at vsnprintf.c:112
112     vsnprintf.c: No such file or directory.

The call stack: System.Private.CoreLib/src/System/Number.Unix.cs: DoubleToNumber Common/src/Interop/Unix/System.Private.CoreLib.Native/Interop.Number.cs: DoubleToString Native/System.Private.CoreLib.Native/pal_cruntime.cpp: CoreLibNative_DoubleToString glibc: vsnprintf.c

The pal_cruntime.cpp is compiled with clang:

/usr/lib/llvm/6/bin/clang++-6.0 --target=arm-linux-gnueabihf  -DBIT32=1 -DDEBUG -DPAL_STDCPP_COMPAT -DPLATFORM_UNIX=1 -D_ARM_ -D_DEBUG -D_TARGET_ARM_=1 -I/dotnet/corert/bin    /obj/Native/Linux.arm.Debug/System.Private.CoreLib.Native -I/dotnet/corert/src/Native/System.Private.CoreLib.Native -I/dotnet/corert/src/Native/inc/unix -isystem /usr/local/include  -std    =c++11 -g   --sysroot=/dotnet/corert/buildscripts/../cross/rootfs/arm --target=arm-linux-gnueabihf --gcc-toolchain=/dotnet/corert/buildscripts/../cross/rootfs/arm/usr -mthumb -mfpu=vfpv3     -Wall -Werror -Wno-invalid-offsetof -Wno-null-arithmetic -Wno-null-conversion -Wno-pragmas -march=armv7-a -Wno-unused-function -fms-extensions -fPIC -fvisibility=hidden -fstack-protector-strong -g -O0 -I/dotnet/corert/src/Native/Common -o CMakeFiles/System.Private.CoreLib.Native.dir/pal_cruntime.cpp.o -c /dotnet/corert/src/Native/System.Private.CoreLib.Native/pal_cruntime.cpp"

The root cause of the issue: jit placed double in R0, R1 registers:

   │0x26362c <S_P_CoreLib_Interop_Sys__DoubleToString+52>   ldr    r0, [sp, #44]   ; 0x2c                                                                                                    │
   │0x26362e <S_P_CoreLib_Interop_Sys__DoubleToString+54>   ldr    r1, [sp, #40]   ; 0x28                                                                                                    │
   │0x263630 <S_P_CoreLib_Interop_Sys__DoubleToString+56>   ldr    r2, [sp, #36]   ; 0x24                                                                                                    │
   │0x263632 <S_P_CoreLib_Interop_Sys__DoubleToString+58>   ldr    r3, [sp, #32]                                                                                                             │
B+>│0x263634 <S_P_CoreLib_Interop_Sys__DoubleToString+60>   bl     0xcfa1c <CoreLibNative_DoubleToString(double, char*, char*, int)>  

Thread 1 "test" hit Breakpoint 1, 0x00263634 in S_P_CoreLib_Interop_Sys__DoubleToString ()
(gdb) info registers 
r0             0x1a     26
r1             0x21     33

but the function CoreLibNative_DoubleToString() (compiled with clang) expects double in d0:

  >│0xcfa1c <CoreLibNative_DoubleToString(double, char*, char*, int)>       push   {r7, lr}                                                                                                  │
   │0xcfa1e <CoreLibNative_DoubleToString(double, char*, char*, int)+2>     mov    r7, sp                                                                                                    │
   │0xcfa20 <CoreLibNative_DoubleToString(double, char*, char*, int)+4>     sub    sp, #48 ; 0x30                                                                                            │
   │0xcfa22 <CoreLibNative_DoubleToString(double, char*, char*, int)+6>     mov    r3, r2                                                                                                    │
   │0xcfa24 <CoreLibNative_DoubleToString(double, char*, char*, int)+8>     mov    r12, r1                                                                                                   │
   │0xcfa26 <CoreLibNative_DoubleToString(double, char*, char*, int)+10>    mov    lr, r0                                                                                                    │
   │0xcfa28 <CoreLibNative_DoubleToString(double, char*, char*, int)+12>    vmov.f64       d16, d0                                                                                           │
   │0xcfa2c <CoreLibNative_DoubleToString(double, char*, char*, int)+16>    vstr   d0, [sp, #40]   ; 0x28  

Then the float value is used as pointer to buffer and application crashed:

#0  0x000cfa38 in CoreLibNative_DoubleToString (value=5.2846989566388942e-308, format=0x1a <error: Cannot access memory at address 0x1a>,
    buffer=0x21 <error: Cannot access memory at address 0x21>, bufferLength=-1090522772) at /dotnet/corert/src/Native/System.Private.CoreLib.Native/pal_cruntime.cpp:12

armel is not affected. @alpencolt

[jkotas]

The root cause of the issue: jit placed double in R0, R1 registers

This would happen if you use armel JIT to generate code for arm. Could you please double check that you are running arm JIT and not armel JIT? I do not think we have any check in place to prevent mismatch like this. It may be worth it to put one in place.

[iarischenko]

. Could you please double check that you are running arm JIT and not armel JIT?

I have two environments, one for arm and one for armel (in docker, other machine). ilc.rsp file for arm starts with:

  1 --targetarch=arm                                                                                                                                                                          
  2 /overlay.arm.Debug/test/test.dll

and ilc.rsp file for armel starts with:

  1 --targetarch=armel
  2 /overlay.armel.Debug/test/test.dll

Compilation string for arm is:

 ./corerun ilc.dll --codegenopt "JitDisasm=*" --codegenopt "AltJitNgen=*" @/overlay.arm.Debug/test/base.ilc.rsp

Compilation string for armel is:

./corerun ilc.dll --codegenopt "JitDisasm=*" --codegenopt "AltJitNgen=*" @/overlay.armel.Debug/test/base.ilc.rsp

Linking for arm

Linking: /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/test/test
clang -target arm-linux-gnueabihf --sysroot=/dotnet/corert/cross/rootfs/arm \
     -B/dotnet/corert/cross/arm /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/test/test.o -o /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/test/test /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/sdk/libbootstrapper.a /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/sdk/libRuntime.a /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/sdk/libSystem.Private.CoreLib.Native.a /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/sdk/libSystem.Private.TypeLoader.Native.a /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/framework/System.Native.a /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/framework/System.Globalization.Native.a -g -Wl,-rpath,'$ORIGIN' -pthread -lstdc++ -ldl -lm -luuid -lrt -fPIC
Linkded: /dotnet/corert/cross/rootfs/x86/overlay.arm.Debug/test/test

Linking for armel

Linking: /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/test/test
clang++-3.9 -target arm-linux-gnueabi --sysroot=/dotnet/corert/cross/rootfs/armel \
     -B/dotnet/corert/cross/armel /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/test/test.o -o /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/test/test /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/sdk/libbootstrapper.a /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/sdk/libRuntime.a /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/sdk/libSystem.Private.CoreLib.Native.a /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/sdk/libSystem.Private.TypeLoader.Native.a /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/framework/System.Native.a /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/framework/System.Globalization.Native.a -g -Wl,-rpath,'$ORIGIN' -pthread -lstdc++ -ldl -lm -luuid -lrt -fPIC
Linkded: /dotnet/corert/cross/rootfs/x86/overlay.armel.Debug/test/test