search

ldc-developers / ldc2.snap

Snap package definition for LDC, the LLVM-based D compiler
11 stars 4 forks source link

Use clang(++) as the C(++) compiler to build LLVM and LDC #112

Closed WebDrake closed 3 years ago

WebDrake commented 4 years ago

Using clang as our compiler both brings the snap package in line with upstream LDC build practices and also opens the door to using LTO when building LDC.

This patch takes advantage of the new package-repositories feature of snapcraft 4 to add the LLVM apt repo, which we can use to install the required clang(++) versions. YAML anchors/aliases are used to store the c-compiler and cxx-compiler choices so that these can be re-used in the 3 different parts. ~We also need to update the CMake option to link against a static C++ stdlib.~

Since this is an experimental feature, we also need to update CI to use the --enable-experimental-package-repositories snapcraft flag.

WebDrake commented 4 years ago

Updated to restore the old static-C++-stdlib option: the newer flag is only supported I believe by LLVM 10+.

diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index 2f15ed5..3e2201b 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -131,6 +131,7 @@ parts:
       cmake ../src/llvm \
         -DCMAKE_INSTALL_PREFIX= \
         -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_CXX_FLAGS=-static-libstdc++ \
         -DCOMPILER_RT_INCLUDE_TESTS=OFF \
         -DCOMPILER_RT_USE_LIBCXX=OFF \
         -DLLVM_BINUTILS_INCDIR=/usr/include \
@@ -140,7 +141,6 @@ parts:
         -DLLVM_ENABLE_TERMINFO=OFF \
         -DLLVM_ENABLE_LIBEDIT=OFF \
         -DLLVM_INCLUDE_TESTS=OFF \
-        -DLLVM_STATIC_LINK_CXX_STDLIB=ON \
         -GNinja
       ninja
       DESTDIR=../install ninja install
WebDrake commented 4 years ago

I'm seeing warnings during the LLVM build:

clang: warning: argument unused during compilation: '-static-libstdc++' [-Wunused-command-line-argument]

This is a little surprising, since that option is used with LDC's llvm-project CI for LLVM 9.

WebDrake commented 4 years ago

The current version fails towards the end of the LLVM build:

2020-07-27T10:33:15.4573612Z [3231/3688] Linking CXX shared library lib/libLTO.so.9
2020-07-27T10:33:15.5235715Z FAILED: : && /usr/bin/clang++-9 -fPIC -static-libstdc++ -fvisibility-inlines-hidden -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wno-noexcept-type -Wnon-virtual-dtor -fdiagnostics-color -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/vsts/work/1/s/parts/llvm/build/./lib  -Wl,-O3 -Wl,--gc-sections  -Wl,--version-script,"/home/vsts/work/1/s/parts/llvm/build/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.9 -o lib/libLTO.so.9 tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64AsmParser.a  lib/libLLVMARMAsmParser.a  lib/libLLVMMipsAsmParser.a  lib/libLLVMMSP430AsmParser.a  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMRISCVAsmParser.a  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMX86AsmParser.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMAArch64CodeGen.a  lib/libLLVMARMCodeGen.a  lib/libLLVMMipsCodeGen.a  lib/libLLVMMSP430CodeGen.a  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMRISCVCodeGen.a  lib/libLLVMWebAssemblyCodeGen.a  lib/libLLVMX86CodeGen.a  lib/libLLVMAVRCodeGen.a  lib/libLLVMAArch64Desc.a  lib/libLLVMARMDesc.a  lib/libLLVMMipsDesc.a  lib/libLLVMMSP430Desc.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMPowerPCDesc.a  lib/libLLVMRISCVDesc.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMX86Desc.a  lib/libLLVMAVRDesc.a  lib/libLLVMAArch64Disassembler.a  lib/libLLVMARMDisassembler.a  lib/libLLVMMipsDisassembler.a  lib/libLLVMMSP430Disassembler.a  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMRISCVDisassembler.a  lib/libLLVMWebAssemblyDisassembler.a  lib/libLLVMX86Disassembler.a  lib/libLLVMAVRDisassembler.a  lib/libLLVMAArch64Info.a  lib/libLLVMARMInfo.a  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVInfo.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMX86Info.a  lib/libLLVMAVRInfo.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMLTO.a  lib/libLLVMMC.a  lib/libLLVMMCDisassembler.a  lib/libLLVMSupport.a  lib/libLLVMTarget.a  lib/libLLVMRISCVUtils.a  lib/libLLVMGlobalISel.a  lib/libLLVMX86Utils.a  lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAArch64Utils.a  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMMCDisassembler.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a  lib/libLLVMipo.a  lib/libLLVMScalarOpts.a  lib/libLLVMVectorize.a  lib/libLLVMBitWriter.a  lib/libLLVMLinker.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMObject.a  lib/libLLVMBitReader.a  lib/libLLVMBitstreamReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMProfileData.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMSupport.a  -lz  -lrt  -ldl  -lpthread  -lm  lib/libLLVMDemangle.a && :
2020-07-27T10:33:15.5243016Z /usr/bin/ld: lib/libLLVMAArch64AsmParser.a(AArch64AsmParser.cpp.o): relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
2020-07-27T10:33:15.5243648Z lib/libLLVMAArch64AsmParser.a: error adding symbols: Bad value
2020-07-27T10:33:15.5244293Z clang: error: linker command failed with exit code 1 (use -v to see invocation)
WebDrake commented 4 years ago

Updated with a tweaked package suite:

diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index 3e2201b..7b0577d 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -29,7 +29,7 @@ package-repositories:
 - type: apt
   deb-types: [deb]
   url: http://apt.llvm.org/$SNAPCRAFT_APT_RELEASE
-  suites: [llvm-toolchain-$SNAPCRAFT_APT_RELEASE]
+  suites: [llvm-toolchain-$SNAPCRAFT_APT_RELEASE-9]
   components: [main]
   key-id: 6084F3CF814B57C1CF12EFD515CF4D18AF4F7421
WebDrake commented 4 years ago

OK, I think I need help with this. I've tried the obvious, of extending the LLVM cmake config with -DCMAKE_CXX_FLAGS='-fPIC -static-libstdc++', and the same problem shows up:

FAILED: : && /usr/bin/clang++-9  -fPIC -fPIC -static-libstdc++ -fvisibility-inlines-hidden -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wno-noexcept-type -Wnon-virtual-dtor -fdiagnostics-color -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/root/parts/llvm/build/./lib  -Wl,-O3 -Wl,--gc-sections  -Wl,--version-script,"/root/parts/llvm/build/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.9 -o lib/libLTO.so.9 tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  lib/libLLVMAArch64AsmParser.a lib/libLLVMARMAsmParser.a lib/libLLVMMipsAsmParser.a lib/libLLVMMSP430AsmParser.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMRISCVAsmParser.a lib/libLLVMWebAssemblyAsmParser.a lib/libLLVMX86AsmParser.a lib/libLLVMAVRAsmParser.a lib/libLLVMAArch64CodeGen.a lib/libLLVMARMCodeGen.a lib/libLLVMMipsCodeGen.a lib/libLLVMMSP430CodeGen.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMRISCVCodeGen.a lib/libLLVMWebAssemblyCodeGen.a lib/libLLVMX86CodeGen.a lib/libLLVMAVRCodeGen.a lib/libLLVMAArch64Desc.a lib/libLLVMARMDesc.a lib/libLLVMMipsDesc.a lib/libLLVMMSP430Desc.a lib/libLLVMNVPTXDesc.a lib/libLLVMPowerPCDesc.a lib/libLLVMRISCVDesc.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMX86Desc.a lib/libLLVMAVRDesc.a lib/libLLVMAArch64Disassembler.a lib/libLLVMARMDisassembler.a lib/libLLVMMipsDisassembler.a lib/libLLVMMSP430Disassembler.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMRISCVDisassembler.a lib/libLLVMWebAssemblyDisassembler.a lib/libLLVMX86Disassembler.a lib/libLLVMAVRDisassembler.a lib/libLLVMAArch64Info.a lib/libLLVMARMInfo.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCInfo.a lib/libLLVMRISCVInfo.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMX86Info.a lib/libLLVMAVRInfo.a lib/libLLVMBitReader.a lib/libLLVMCore.a lib/libLLVMLTO.a lib/libLLVMMC.a lib/libLLVMMCDisassembler.a lib/libLLVMSupport.a lib/libLLVMTarget.a lib/libLLVMRISCVUtils.a lib/libLLVMGlobalISel.a lib/libLLVMX86Utils.a lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMSelectionDAG.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMARMDesc.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMMCDisassembler.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a lib/libLLVMipo.a lib/libLLVMScalarOpts.a lib/libLLVMVectorize.a lib/libLLVMBitWriter.a lib/libLLVMLinker.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a lib/libLLVMBitReader.a lib/libLLVMBitstreamReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a lib/libLLVMRemarks.a lib/libLLVMSupport.a -lz -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a -Wl,-rpath,"\$ORIGIN/../lib" && :
/usr/bin/ld: lib/libLLVMSupport.a(regcomp.c.o): relocation R_X86_64_32 against `.bss.nuls' can not be used when making a shared object; recompile with -fPIC
lib/libLLVMSupport.a: error adding symbols: Bad value
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@JohanEngelen @kinke do you have any idea what could be the problem here? I can't see any obvious differences between what I've got set up and the cmake config used in llvm-project.

JohanEngelen commented 4 years ago

Sorry, don't know what's going on. Linker errors are always fun...

WebDrake commented 4 years ago

Indeed! Thanks for taking a look, anyway :-)

I realized there is a LVVM_ENABLE_PIC option: I'm trying using that now, but I assume it is ON by default anyway. The other thing I came across was an LLVM developer conversation where they discuss having a LIBCXXABI_ENABLE_PIC option for use with the static C++ lib. Not sure if they are talking about the same thing, but I'll dive deeper and see if it can help.

WebDrake commented 4 years ago

OK, LIBCXXABI_ENABLE_PIC is not recognized by the LLVM build. I'll keep digging ...

WebDrake commented 4 years ago

Rebased on 1.20 and removed the -static-libstdc++ option:

diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index 7b0577d..6b1f719 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -41,6 +41,7 @@ parts:
     source: https://github.com/ldc-developers/ldc.git
     source-tag: &ldc-version v1.20.1
     source-type: git
+    source-depth: 1
     plugin: cmake
     override-build: |
       cmake ../src \
@@ -95,6 +96,7 @@ parts:
     source: https://github.com/ldc-developers/ldc.git
     source-tag: *ldc-version
     source-type: git
+    source-depth: 1
     plugin: cmake
     override-build: |
       cmake ../src \
@@ -126,12 +128,12 @@ parts:
     source: https://github.com/ldc-developers/llvm-project.git
     source-tag: ldc-v9.0.1
     source-type: git
+    source-depth: 1
     plugin: cmake
     override-build: |
       cmake ../src/llvm \
         -DCMAKE_INSTALL_PREFIX= \
         -DCMAKE_BUILD_TYPE=Release \
-        -DCMAKE_CXX_FLAGS=-static-libstdc++ \
         -DCOMPILER_RT_INCLUDE_TESTS=OFF \
         -DCOMPILER_RT_USE_LIBCXX=OFF \
         -DLLVM_BINUTILS_INCDIR=/usr/include \
WebDrake commented 4 years ago

May be worth rebasing/retargeting on 1.19 as this is the first upstream package where LTO was used.

WebDrake commented 4 years ago

Rebased on 1.19 and added a small note to the commit message about the -static-libstdc++ change.

WebDrake commented 4 years ago

Switching back to 1.20 to see if it makes a difference to the testsuite outcomes.

WebDrake commented 4 years ago

This is the test that was failing for 1.19:

2020-09-06T22:50:14.2107797Z 3356: /home/vsts/work/1/s/parts/ldc/build/bin/ldmd2 -conf= -m32 -Irunnable -g -link-defaultlib-debug -L-lstdc++ -L--no-demangle  -od../../../../build/dmd-testsuite-debug_32/runnable -of../../../../build/dmd-testsuite-debug_32/runnable/cppa_0  runnable/cppa.d ../../../../build/dmd-testsuite-debug_32/runnable/cppb.cpp.o 
2020-09-06T22:50:14.2109137Z 3356: ../../../../build/dmd-testsuite-debug_32/runnable/cppa_0
2020-09-06T22:50:14.2110048Z 3356: core.exception.AssertError@runnable/cppa.d(256): Assertion failure
2020-09-06T22:50:14.2110863Z 3356: ----------------
2020-09-06T22:50:14.2111468Z 3356: runtime.d:836 [0x807c2e7]
2020-09-06T22:50:14.2112026Z 3356: runtime.d:780 [0x807ba2a]
2020-09-06T22:50:14.2112592Z 3356: dmain2.d:300 [0x805ac6c]
2020-09-06T22:50:14.2113342Z 3356: deh.d:24 [0x80971f6]
2020-09-06T22:50:14.2113899Z 3356: dwarfeh.d:324 [0x805b9d3]
2020-09-06T22:50:14.2114452Z 3356: exception.d:429 [0x8051974]
2020-09-06T22:50:14.2115028Z 3356: exception.d:595 [0x8052070]
2020-09-06T22:50:14.2115584Z 3356: cppa.d:256 [0x804adb5]
2020-09-06T22:50:14.2116118Z 3356: ??:? [0x804f56e]
2020-09-06T22:50:14.2116632Z 3356: cppa.d:261 [0x804af1d]
2020-09-06T22:50:14.2117173Z 3356: cppa.d:1620 [0x804eb9d]
2020-09-06T22:50:14.2117703Z 3356: dmain2.d:587 [0x805b41b]
2020-09-06T22:50:14.2119487Z 3356: dmain2.d:548 [0x805b1d5]
2020-09-06T22:50:14.2119816Z 3356: dmain2.d:587 [0x805b31c]
2020-09-06T22:50:14.2120086Z 3356: dmain2.d:548 [0x805b1d5]
2020-09-06T22:50:14.2120372Z 3356: dmain2.d:607 [0x805b01a]
2020-09-06T22:50:14.2120637Z 3356: dmain2.d:392 [0x805ad7b]
2020-09-06T22:50:14.2120928Z 3356: entrypoint.d:35 [0x804edce]
2020-09-06T22:50:14.2121218Z 3356: ??:? __libc_start_main [0xf7aba646]
2020-09-06T22:50:14.2121499Z 3356: ??:? [0x804a280]
2020-09-06T22:50:14.2121724Z 3356: i = 1
2020-09-06T22:50:14.2121923Z 3356: j = 2
2020-09-06T22:50:14.2122133Z 3356: k = 3
2020-09-06T22:50:14.2122325Z 3356: i = 1
2020-09-06T22:50:14.2122532Z 3356: j = 2
2020-09-06T22:50:14.2122726Z 3356: k = 3
2020-09-06T22:50:14.2122929Z 3356: i = 1
2020-09-06T22:50:14.2123136Z 3356: j = 2
2020-09-06T22:50:14.2123329Z 3356: k = 3
2020-09-06T22:50:14.2123556Z 3356: this = 0xf799f000
2020-09-06T22:50:14.2123774Z 3356: i = 4
2020-09-06T22:50:14.2123967Z 3356: j = 5
2020-09-06T22:50:14.2124175Z 3356: k = 6
2020-09-06T22:50:14.2124385Z 3356: this = 0x959fa80
2020-09-06T22:50:14.2124630Z 3356: D.bar: i = 9
2020-09-06T22:50:14.2124859Z 3356: D.bar: j = 10
2020-09-06T22:50:14.2125087Z 3356: D.bar: k = 11
2020-09-06T22:50:14.2125328Z 3356: F.bar: i = 11
2020-09-06T22:50:14.2125555Z 3356: F.bar: j = 12
2020-09-06T22:50:14.2125795Z 3356: F.bar: k = 13
2020-09-06T22:50:14.2125997Z 3356: 
2020-09-06T22:50:14.2126176Z 3356: 
2020-09-06T22:50:14.2126416Z 3356: ==============================
2020-09-06T22:50:14.2126771Z 3356: Test runnable/cppa.d failed: expected rc == 0, exited with rc == 1
2020-09-06T22:50:14.2127082Z 3356: 
2020-09-06T22:50:14.2127855Z 3356: Makefile:372: recipe for target '../../../../build/dmd-testsuite-debug_32/runnable/cppa.d.out' failed

I'll dive into it later and see if I can work out why.

WebDrake commented 4 years ago

This is the test that was failing for 1.19:

The same test fails for 1.20. It appears to be this one:

struct S13956
{
}

extern(C++) void func13956(S13956 arg0, int arg1, int arg2, int arg3, int arg4, int arg5, int arg6);

extern(C++) void check13956(S13956 arg0, int arg1, int arg2, int arg3, int arg4, int arg5, int arg6)
{
    assert(arg0 == S13956());
    assert(arg1 == 1);
    assert(arg2 == 2);
    assert(arg3 == 3);
    assert(arg4 == 4);
    assert(arg5 == 5);
    assert(arg6 == 6);
}

void test13956()
{
    func13956(S13956(), 1, 2, 3, 4, 5, 6);
}

... which is a test for this issue https://issues.dlang.org/show_bug.cgi?id=13956, checking that empty structs are never allocated a register.

kinke commented 4 years ago

Nothing really to worry about, clang and gcc just don't seem to agree on the 32-bit x86 ABI wrt. these empty structs (https://github.com/ldc-developers/ldc/blob/032b492bcbc77625866e4a86b3cbc990a74bfb7a/gen/abi-x86.cpp#L207-L211). Azure uses clang only for building LDC itself, not for the later testing steps (where CC and CXX are in their vanilla state).

WebDrake commented 4 years ago

Azure uses clang only for building LDC itself, not for the later testing steps (where CC and CXX are in their vanilla state).

Ahh, thanks. I'll see if I can switch back to gcc/g++ for the test suite run.

WebDrake commented 4 years ago

I've tweaked to reset CC and CXX when running the test suite:

CC=gcc CXX=g++ ctest --output-on-failure --verbose -E "std\.net\.curl|std\.process|lit-tests"
WebDrake commented 4 years ago

I'm worried about why -static-libstdc++ causes a problem, though: it's used in the upstream LDC CI, and there are good reasons to suppose it might matter for the snap package (for the same cross-distro-support reason it matters in the linux binary packages).

kinke commented 4 years ago

I guess this is due to a different binutils version then - Azure uses the (at the time, latest) Ubuntu 16.04 docker image and its binutils some Ubuntu-16.04-based image, see https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu1604-README.md.

WebDrake commented 4 years ago

I'm starting to ponder whether a better long-term option might not be to derive the snap packages from upstream builds. I may try to put together something with that in mind.

WebDrake commented 3 years ago

Closing as superseded by more recent changes (snap package now just wraps upstream builds).