search

ClangBuiltLinux / continuous-integration

Continuous integration of latest Linux kernel with daily build of Clang & LLVM tools
https://travis-ci.com/ClangBuiltLinux/continuous-integration
Apache License 2.0
44 stars 18 forks source link

investigate cross-compilation with LLVM tools and without GNU binutils #73

Open tpimh opened 5 years ago

tpimh commented 5 years ago

I'm not sure that cross compilation is possible with llvm tools at this point. I will research the topic and post my findings in this thread.

nickdesaulniers commented 5 years ago

I don't follow. All of Android is cross compiled via LLVM. Which parts are you referring to specifically? LLVM uses a combination of -triple <target triple> and --sysroot <path to cross crt0.o and friends> .

tpimh commented 5 years ago

Right now we are installing binutils-aarch64-linux-gnu, binutils-arm-linux-gnueabi, binutils-powerpc64le-linux-gnu. These packages provide tools specific for each platform with respective cross-prefixes. My idea is to replace these tools with the ones from llvm-8 package with "llvm-" prefix. For example, instead of using objcopy, aarch64-linux-gnu-objcopy, arm-linux-gnueabi-objcopy, powerpc64le-linux-gnu-objcopy use llvm-objcopy for all platforms.

nickdesaulniers commented 5 years ago

@rupprecht is looking in to such incompatibilties as well. https://github.com/ClangBuiltLinux/linux/issues?q=is%3Aopen+is%3Aissue+label%3Allvm-tools is our list of known issues (separate from Clang, LLD, or integrated-as).

tpimh commented 5 years ago

In order to make sure there is no binutils installed I do dpkg -r --force-depends binutils binutils-common binutils-x86-64-linux-gnu libbinutils for x86_64, then apt remove binutils-* for all other platforms. At the moment it's not possible to build the kernel with clang's integrated as, so this is not very useful. However, it's possible to use llvm-ar, llvm-nm, llvm-objdump, llvm-objcopy and llvm-strip.

tpimh commented 5 years ago

After very little modification (replacing about 10 lines of unsupported assembly with nops), I finally have built an x86_64 defconfig kernel without using GNU binutils. It obviously should not be bootable, but it was linked successfully. Currently the issues are the following:

There may be other issues like invalid code generation, but they are very unlikely to be detected until a kernel is compiled without stubs.

nickdesaulniers commented 5 years ago

cc @gwelymernans

tpimh commented 4 years ago

Hooray! I have successfully build an arm32_v5 kernel without using GNU binutils: travis build log.

nathanchance commented 4 years ago

Awesome! Something like this should reveal if everything is hermetic:

https://github.com/nathanchance/continuous-integration/commit/0e2a26e3ea3f028fc32fd91abf072d50175cce25

I suspect we are still implicitly using ld as the host linker when HOSTCC=clang for certain things, as that is what I ran into:

https://github.com/nathanchance/continuous-integration/commit/730028d23f139ec0a318337fbe2444dc7fdc485f

https://github.com/ClangBuiltLinux/linux/issues/479

Otherwise, I'd expect only LLVM tools to be used.

tpimh commented 4 years ago

Thank you very much for your suggestions. I was sure that GNU binutils were not used during the build, but still wanted to remove them. Turned out the build was not actually hermetic!

nickdesaulniers commented 4 years ago

great find, filed https://github.com/ClangBuiltLinux/linux/issues/766

tpimh commented 4 years ago

arm32_v5 successfull build with total 4 patches: https://travis-ci.com/ClangBuiltLinux/continuous-integration/jobs/254762048 x86_64 is not so successful; missing readelf, producing a lot of objtool warnings and not booting, definitely needs more investigation: https://travis-ci.com/ClangBuiltLinux/continuous-integration/jobs/254762056

nickdesaulniers commented 4 years ago

with total 4 patches

Which 4/where?

missing readelf,

arch/x86/boot/compressed/Makefile hardcodes readelf, similar to the nm issue on arm.

tpimh commented 4 years ago

These are the patches required for successful build for arm32_v5:

tpimh commented 4 years ago

Right now x86_64 boots fine. I also have compiled a bootable ARM64 kernel with the following workarounds: