Open Quuxplusone opened 15 years ago
Quuxplusone
commented
15 years ago Some other notes on what I had to do to build the kernel with llvm-gcc:
- the 16bit bootcode arch/x86/boot has to be built with gcc
- a workaround for a bug in lib/vsprintf.c (undefined behaviour: pointer
arithmetic that overflows), I used RELOC_HIDE macro for this:
/* Make sure end is always >= buf */
if (end < buf) {For reference I attached the 2 original patches I used when building the kernel with llvm-gcc: http://llvm.org/bugs/attachment.cgi?id=2896 http://llvm.org/bugs/attachment.cgi?id=2897
They may be useful when building with clang too.
Quuxplusone
commented
14 years ago An update:
compiling Linux 2.6.31-rc9 (with attached patches applied) results in a kernel that boots in qemu, if arch/x86/boot and arch/x86/vdso is compiled with gcc.
If boot is compiled with llvm, then kernel doesn't boot at all, and if vdso is
compiled with llvm there's a message during the boot that
VDSO: variable vgetcpu_mode broken
VDSO: variable vsyscall_gtod_data broken
What I used:
llvm-gcc, llvm, clang from release2.6 branch, with r81342 patch applied to llvm-
gcc.
llvm-gcc compiled Linux boots, clang compiled Linux only boots up to the point
it launches init. After that it shows the login prompt, but its broken, won't
let me login, and also init didn't start any of the other boot scripts (like
the script that launches udev).
Boot messages diff is attached.Attached 0001-inline-asm-patch-for-llvm.patch (4269 bytes, text/plain): patch 1/2 used with llvm-gcc
Quuxplusone
commented
14 years ago Attached 0001-1-2-llvm-gcc.patch (4326 bytes, text/x-diff): patch to fix inline asm constraint in Linux to be compilable by llvm-gcc
Quuxplusone
commented
15 years ago Attached 0002-workaround-undefined-behaviour.patch (700 bytes, text/plain): patch needed for llvm-gcc and clang to workaround lack of support for -fno-strict-overflow/-fwrapv
Quuxplusone
commented
15 years ago Attached .config (50331 bytes, application/octet-stream): .config that builds
Quuxplusone
commented
14 years ago Attached .config (52655 bytes, application/octet-stream): .config that builds (2.6.31-rc8)
Quuxplusone
commented
14 years ago Attached 0003-llvm-gcc-boot-vdso-compiled-with-gcc.patch (1861 bytes, text/x-diff): compile boot and vdso with gcc when buliding with llvm-gcc
Quuxplusone
commented
15 years ago Attached clang-linux.patch (6362 bytes, text/plain): patch to Linux kernel needed
Quuxplusone
commented
14 years ago Attached 0001-clang-makefile-hacks-and-workaround-for-address-of-l.patch (7361 bytes, text/x-diff): patch to Linux kernel to build with clang
Quuxplusone
commented
14 years ago llvm-gcc needs these patches to Linux kernel:
http://llvm.org/bugs/attachment.cgi?id=2897
http://llvm.org/bugs/attachment.cgi?id=3483
http://llvm.org/bugs/attachment.cgi?id=3485
clang needs these patches to Linux kernel:
http://llvm.org/bugs/attachment.cgi?id=2897
http://llvm.org/bugs/attachment.cgi?id=3486
boot is not built correctly by llvm because it relies on .code16gcc.
I haven't investigated why vdso is not built properly.I have successfully built a Linux 2.6.36 kernel with clang. It can self host itself, boot to runlevel 5 successfully, SMP works, X works, networking works, etc (posting this message from a machine running a self-hosted linux kernel built with clang :)). There's still a lot that's buggy, but clang can definitely compile Linux!
Quuxplusone
commented
13 years ago (In reply to comment #13)
> I have successfully built a Linux 2.6.36 kernel with clang. It can self host
> itself, boot to runlevel 5 successfully, SMP works, X works, networking works,
> etc (posting this message from a machine running a self-hosted linux kernel
> built with clang :)). There's still a lot that's buggy, but clang can
> definitely compile Linux!
I'll post to the ML with more details and stats. In the morning I should have a
chance to put together the patches and instructions for how to do this.As far as I know, all functionality blocking bugs are fixed for building the linux kernel. The one thing that is missing is a warning (-Wframe-larger-than=2048), which is tracked by PR4072 (see the PR for the menuconfig option to disable the warning).
Edwin (or anyone else interested!), it would be great to see if you could build the kernel and if you run into any other issues. Since this bug was filed, we switched to the integrated assembler, so I wouldn't be too shocked if there were other minor things, but they should be easy to fix.
The LLVM 2.9 release branches on March 6th, it would be great to be able to say that it can build a good kernel, as it would be great to get more of the linux kernel development community off of GCC :-)
Quuxplusone
commented
13 years ago (In reply to comment #15)
> As far as I know, all functionality blocking bugs are fixed for building the
> linux kernel. The one thing that is missing is a warning
> (-Wframe-larger-than=2048), which is tracked by PR4072 (see the PR for the
> menuconfig option to disable the warning).
>
> Edwin (or anyone else interested!), it would be great to see if you could
build
> the kernel and if you run into any other issues. Since this bug was filed, we
> switched to the integrated assembler, so I wouldn't be too shocked if there
> were other minor things, but they should be easy to fix.
>
> The LLVM 2.9 release branches on March 6th, it would be great to be able to
say
> that it can build a good kernel, as it would be great to get more of the linux
> kernel development community off of GCC :-)
Hey Chris!
I've been using a Clang compiled Linux kernel on my Macbook 5.1 since late
October of 2010. Three of my other machines also have working Linux kernels
built with Clang (they do not primarily run said kernels, as they are my
"sane", non-experimental environments). I've had to compile tricky,
closed-source kernel drivers (and tricky, open-source kernel drivers). I still
haven't had a chance to submit my hacks to Clang/LLVM/Linux (hacks to the
kernel are minimal), but I believe most of them have been written by someone
else and placed into the kernel tree. I really wish I could take more of an
active role, but I'm very busy at work these days.
I believe there were some news posts and a list post that I myself made when I
first got it working. There seemed to be some interest, but no one stepped
forward expressing a willingness to assist in cleaning up the necessary
patches/testing the kernel/providing instructions for building/distributing
information on Clang/Linux. Two other parties (PAX security team, Alp
Toker/Nuanti) were incredibly helpful in getting a Clang built kernel working,
but none of us had the time to spearhead the organized effort that would be
needed to make turn this experimental first-step into a viable, production-ready
alternative for Linux kernel compilation.
I, admittedly, have been neglectful in getting my patches into Clang, and I
sincerely apologize for that. This is the sort of thing that really interests
me, but further progress (aka going from "hey, some kid hacked together the
last few pieces of this puzzle" to serious alternative to GNU GCC) is very
daunting, and I'm not going to start working on something that I know I cannot
tackle on my own. If I continued work on this (even insofar as just cleaning up
my patches and sending them upstream), I knew I would end up getting sucked in
:P.
I do not feel that I have the experience or discipline to take a central role
in further development here, irregardless of how busy I am. If another party
or parties is actively working (or plans to actively work) on Clang/Linux
(open source working, I mean), please contact me, as I would like to
participate and share what I have learned/my experience. I think it'd be great
if a senior LLVM or Clang developer would start an LLVM subproject for Clang
compiled Linux. Getting a Linux kernel developer involved would also be great
(I am neither).
My email is admin-at-thefireflyproject.us or blelbach-at-cct.lsu.edu.(In reply to comment #15)
> Edwin (or anyone else interested!), it would be great to see if you could
build
> the kernel and if you run into any other issues.
Hi Chris,
I think these are the most important clang bugs:
ICE:
http://llvm.org/bugs/show_bug.cgi?id=9252
http://llvm.org/bugs/show_bug.cgi?id=9262
ABI / struct layout:
http://llvm.org/bugs/show_bug.cgi?id=9253
llc -O1 creates invalid module:
http://llvm.org/bugs/show_bug.cgi?id=9259
more GNU array range designators:
http://llvm.org/bugs/show_bug.cgi?id=9257
http://llvm.org/bugs/show_bug.cgi?id=9261
inline asm:
http://llvm.org/bugs/show_bug.cgi?id=9258
It is debatable whether the other bugs I opened are bugs in Linux kernel or
Clang, depending if you consider that the C standard allows you to break the
code, or if you want a useful C language where you at least warn that you'll
break the code, and preferably don't break it if you can.
There are also some bugs in clang's warnings, but I didn't mark those as
blocking this bug. They are way too noisy, I had to use this:
-Wno-unused-value -Wno-array-bounds -Wno-unused-variable -Qunused-arguments -Wno-initializer-overrides
And we should do something about #9254, since now it also prevents iptables
from being compiled (it only used to be crypto). Which probably means someone
will have to patch the kernel if clang will not implement this:
http://llvm.org/bugs/show_bug.cgi?id=9254hi, is there a git branch of linux somewhere that include the patchs for
compiling the kernel with clang ?
If not it would be great to setup one so that persons interested on the subject
share more easily their work :)(In reply to comment #18)
> hi, is there a git branch of linux somewhere that include the patchs for
> compiling the kernel with clang ?
> If not it would be great to setup one so that persons interested on the
subject
> share more easily their work :)
I have my patches attached to various bugreports, I can cleanup my local branch
with patches and push it somewhere.I am trying to compile an unmodified linux kernel source tree with the latest
svn clang, and I get a bunch of the error below, on top of a fair number of
warnings. Is there some patch to the kernel that I should be applying before
attempting this build? This is just a wild guess, but is the problem with the
assembly below that it's using %eax for a byte-level instruction? Should it be
'movl' rather than 'movb'?
init/main.c:511:12: error: invalid operand for instruction
int cpu = smp_processor_id();
^
In file included from init/main.c:13:
In file included from include/linux/module.h:13:
In file included from include/linux/kmod.h:22:
In file included from include/linux/gfp.h:7:
In file included from include/linux/topology.h:33:
include/linux/smp.h:174:29: note: instantiated from:
# define smp_processor_id() raw_smp_processor_id()
^
In file included from init/main.c:13:
In file included from include/linux/module.h:13:
In file included from include/linux/kmod.h:22:
In file included from include/linux/gfp.h:7:
In file included from include/linux/topology.h:33:
In file included from include/linux/smp.h:37:
/home/kevin/linux/linux-2.6/arch/x86/include/asm/smp.h:175:33: note:
instantiated from:
#define raw_smp_processor_id() (percpu_read(cpu_number))
^
In file included from init/main.c:13:
In file included from include/linux/module.h:9:
In file included from include/linux/list.h:7:
In file included from include/linux/prefetch.h:14:
In file included from /home/kevin/linux/linux-
2.6/arch/x86/include/asm/processor.h:15:
In file included from /home/kevin/linux/linux-
2.6/arch/x86/include/asm/current.h:5:
/home/kevin/linux/linux-2.6/arch/x86/include/asm/percpu.h:360:27: note:
instantiated from:
#define percpu_read(var) percpu_from_op("mov", var, "m" (var))
^
init/main.c:511:12: note: instantiated from:
int cpu = smp_processor_id();
^
In file included from init/main.c:13:
In file included from include/linux/module.h:13:
In file included from include/linux/kmod.h:22:
In file included from include/linux/gfp.h:7:
In file included from include/linux/topology.h:33:
include/linux/smp.h:174:29: note: instantiated from:
# define smp_processor_id() raw_smp_processor_id()
^
In file included from init/main.c:13:
In file included from include/linux/module.h:13:
In file included from include/linux/kmod.h:22:
In file included from include/linux/gfp.h:7:
In file included from include/linux/topology.h:33:
In file included from include/linux/smp.h:37:
/home/kevin/linux/linux-2.6/arch/x86/include/asm/smp.h:175:33: note:
instantiated from:
#define raw_smp_processor_id() (percpu_read(cpu_number))
^
In file included from init/main.c:13:
In file included from include/linux/module.h:9:
In file included from include/linux/list.h:7:
In file included from include/linux/prefetch.h:14:
In file included from /home/kevin/linux/linux-
2.6/arch/x86/include/asm/processor.h:15:
In file included from /home/kevin/linux/linux-
2.6/arch/x86/include/asm/current.h:5:
/home/kevin/linux/linux-2.6/arch/x86/include/asm/percpu.h:360:42: note:
instantiated from:
#define percpu_read(var) percpu_from_op("mov", var, "m" (var))
^
<inline asm>:1:22: note: instantiated into assembly here
movb %gs:cpu_number,%eax
^Looks like a kernel bug, "movb %gs:cpu_number,%eax" is invalid because you can't "movb" into a 32-bit register.
Quuxplusone
commented
13 years ago It's a bit more complicated than that; see bug 9322.
Quuxplusone
commented
13 years ago 4898 is an optimization hint, it doesn't block this.
Quuxplusone
commented
13 years ago Some of us have gotten together and started working on this collaboratively, https://github.com/lll-project.
Quuxplusone
commented
11 years ago We have a project called LLVMLinux which brings together all the various efforts around compiling the Linux kernel with Clang. Our project has also recently merged with the lll-project team. Our intentions are to submit patches to Clang/LLVM project as well as the Linux kernel in order to make it possible to fully compile the Linux kernel with Clang. You can learn more at our website: http://llvm.linuxfoundation.org/
Quuxplusone
commented
11 years ago Added 15676, a new bug which is a segfault when cross-compiling the linux Kernel for ARM.
Quuxplusone
commented
6 years ago This is about my successful experience in bringing up Ubuntu 17.10 x86_64 on
the clang-built Linux 4.15.7 kernel(latest stable as of now) and clang-built
proprietary Nvidia 390.25(latest as of now) driver for my Nvidia GTX GeForce
860M graphics card. I wanted to share the same here for any similarly
interested people out here and also as an update for the progress of
clang/Linux kernel compatibility as I have used, researched and observed. I
have shared a complete report on this in the below places --
"Success : Ubuntu 17.10 x86_64 on LLVM/clang-built kernel with Nvidia 390.25
driver" :
1) https://ubuntuforums.org/showthread.php?t=2386423,
2) https://askubuntu.com/questions/1012186/success-ubuntu-17-10-x86-64-on-llvm-
clang-built-kernel-with-llvm-clang-built-n
3) https://www.quora.com/What-is-the-state-of-the-art-of-compiling-the-Linux-
Kernel-using-LLVM-Clang/answer/Raghavan-Santhanam
Cheers.Hello,
This is about successful experience of bringing-up Android ARM(32-bit)
LLVM/clang-built Linux kernel on Nexus 5(hammerhead) Android Smartphone.
I wanted to share the same here for the interested people. Though I saw the
reports of AARCH64 clang-built Android kernel, the information over there
didn't help in my case of ARM(32-bit) - so I have done my ARM(32-bit) clang-
built kernel's build and bring-up work from *scratch*.
In this effort of mine, I have successfully brought up the Android ARM(32-bit)
kernel built with 4 different LLVM/clang toolchains/versions :
1) Android clang version 3.8.256229 (based on LLVM 3.8.256229) from "android-
ndk-r13b"
2) Android (4691093 based on r316199) clang version 6.0.2
(https://android.googlesource.com/toolchain/clang
183abd29fc496f55536e7d904e0abae47888fc7f)
(https://android.googlesource.com/toolchain/llvm
34361f192e41ed6e4e8f9aca80a4ea7e9856f327) (based on LLVM 6.0.2svn) from
"android-ndk-r17"
3) Flash clang version 7.0.332826 (https://git.llvm.org/git/clang
4029c7ddda99ecbfa144f0afec44a192c442b6e5) (https://git.llvm.org/git/llvm
1181c40e0e24e0cca32e2609686db1f14151fc1a) (based on LLVM 7.0.332826)
4) Snapdragon LLVM ARM Compiler 4.0.2 for Android NDK (based on llvm.org 4.0+) -
clang version 4.0.2 for Android NDK from "Qualcomm snapdragon-llvm-4.0.2-linux64.tar.gz"
Complete reports on this research project of mine can be found here :
"Success: Bring-up of LLVM/clang-built Linux ARM(32-bit) kernel for Android -
Nexus 5" - https://ubuntuforums.org/showthread.php?t=2394035
Cheers.Hello,
This is an update to my above initial report of x86_64 LLVM/clang-built kernel
for my system hardware running Ubuntu 17.10 x86_64. After having used the
latest LLVM/clang versions for my above Android ARM(32-bit) Linux kernel, I was
interested in trying out the same for my system hardware running Ubuntu 17.10
x86_64. So, I tried using clang-6.0 as well as the clang-7.0(developmental
version/built from LLVM source - same as the one used for my recent above
reported Android ARM(32-bit) Linux kernel's case) in separate attempts. With
additional effort, I was able to bring-up Ubuntu 17.10 x86_64 on my system
hardware with both of these kernels individually, along with other kernel
driver modules including that of NVIDIA, VirtualBox, VMware Player, etc with
all of them fully working on my system hardware - for the hypervisors(their
kernel modules, to be precise) with all the so far clang-built host kernels,
verified using a fully working Ubuntu 17.10 x86_64 guest - for the NVIDIA case,
have verified the availability of graphics hardware acceleration, Ubuntu's
system/GUI stability, etc in all of the clang-built kernel cases - so far there
have been no issues.
Details of this above update(using clang-6.0 and clang-7.0 for building and
bringing-up the kernel) can be found in my original post towards the end of it
here :
"Success : Ubuntu 17.10 x86_64 on LLVM/clang-built kernel with NVIDIA,
VirtualBox, VMware Player modules" -
https://ubuntuforums.org/showthread.php?t=2386423.
Here are the mappings of the updates in the above thread, onto the kernel
versions, all for Ubuntu 17.10 x86_64 on my system hardware :
1) Update 4 and earlier/clang-5.0 : Linux v4.17.0(and versions as early as
v4.15.7 - my first successful fully working clang-built kernel for Ubuntu 17.10
x86_64)
2) Update 5/clang-6.0 : Linux 4.17.2(latest stable, as of now)
3) Update 6/clang-7.0(dev-version) : Linux 4.17.2(latest stable, as of now)
As a relevant update, in addition to VirtualBox(Version 5.2.8 r120774) kernel
modules using LLVM/clang, I have been successful
in building and bringing-up the VirtualBox userspace application itself using
LLVM/clang(++). Details of the same can be found here :
"Success : VirtualBox(clang-built): Ubuntu 17.10 x86_64(clang-built
kernel/modules)" -
https://ubuntuforums.org/showthread.php?t=2389838
As another relevant update, in addition to building VMware Player(14.1.2 build-
8497320) kernel modules using LLVM/clang, I have been able to successfully
bring-up
VMware Player application with the host clang-built kernel - this needed extra
effort as VMware Player expects the host kernel to be gcc-built.
Details of the same can be found here :
"Success : Ubuntu 17.10 x86_64 - VMware Player - clang-built kernel and kernel
modules" -
https://ubuntuforums.org/showthread.php?t=2388806
Cheers.Hello,
This is an update to my above successful bringing-up LLVM/clang-built ARM(32-
bit) Android Linux kernel for different Android(Marshmallow to Nougat to Oreo)
on Nexus 5 Android Smartphone.
As of now, a total of 10 Android custom ROMS including AOSP(latest factory
image) have been successfully brought-up with their respective kernel sources
built using LLVM/clang. This includes 9 more(AOSP + 8 other popular Android
custom ROMs) along with the initial Resurrection Remix Android custom ROM.
Below is the complete list of these 10 Android ROMs including AOSP.
1) AOSP(Android Open Source Project)/Factory image :
https://developers.google.com/android/images#hammerhead
2) Resurrection Remix : http://www.resurrectionremix.com/
3) LineageOS : https://lineageos.org/
4) OmniROM : https://www.omnirom.org/
5) AospExtended : https://www.aospextended.com/
6) Nitrogen OS : https://sourceforge.net/projects/nitrogen-project/
7) Dirty Unicorn : https://dirtyunicorns.com/
8) AOKP : http://aokp.co/
9) Unlegacy Android : https://www.unlegacy-android.org/projects/unlegacy-
android/wiki
10) Paranoid Android : http://paranoidandroid.co/
Here's my final and complete report on this research of mine here :
https://androidforums.com/threads/success-bring-up-of-llvm-clang-built-linux-arm-32-bit-kernel-for-android-nexus-5.1283026/
and
https://victorymindtalk.blogspot.com/2018/08/success-bring-up-of-llvmclang-built.html
Cheers.Hello,
As of now, I have completed some more successful LLVM/clang-built kernel bring-up works for other popular Linux distributions. Below are the details.
1) LLVM/clang-8.0(dev) - "Success : CentOS 7 x86_64 on LLVM/clang-built kernel with LLVM/clang-built : NVIDIA, VirtualBox, VMware Player modules" - https://victorymindtalk.blogspot.com/2018/09/success-centos-7-x8664-on-llvmclang.html
2) LLVM/clang-7.0(dev) - "Success : Fedora 28 x86_64 on LLVM/clang-built kernel with LLVM/clang-built : NVIDIA, VirtualBox, VMware Player modules" - https://victorymindtalk.blogspot.com/2018/08/success-fedora-28-x8664-on-llvmclang.html
Some of the previously mentioned ..ubuntuforums.. links seem to not work anymore. So, I have republished all of my previous related works as under :
a) "Success : Ubuntu 17.10 x86_64 on LLVM/clang-built kernel with NVIDIA, VirtualBox, VMware Player modules" - https://victorymindtalk.blogspot.com/2018/08/success-ubuntu-17.html
b) "Success : Ubuntu 17.10 x86_64 - VMware Player - LLVM/clang-built kernel and kernel modules" - https://victorymindtalk.blogspot.com/2018/08/success-ubuntu-1710-x8664-vmware-player.html
c) "Success : VirtualBox(LLVM/clang-built): Ubuntu 17.10 x86_64(LLVM/clang-built kernel/modules)" - https://victorymindtalk.blogspot.com/2018/08/success-virtualboxllvmclang-built.html
Cheers
clang-linux.patch(6362 bytes, text/plain)0001-inline-asm-patch-for-llvm.patch(4269 bytes, text/plain)0002-workaround-undefined-behaviour.patch(700 bytes, text/plain).config(50331 bytes, application/octet-stream)0001-1-2-llvm-gcc.patch(4326 bytes, text/x-diff).config(52655 bytes, application/octet-stream)0003-llvm-gcc-boot-vdso-compiled-with-gcc.patch(1861 bytes, text/x-diff)0001-clang-makefile-hacks-and-workaround-for-address-of-l.patch(7361 bytes, text/x-diff)This is a meta-bug, other bugs that prevent building the Linux kernel with clang will be made to depend on it.
We should keep this bug open, until someone boots a kernel built with clang :)