search

ClangBuiltLinux / linux

Linux kernel source tree
Other
241 stars 14 forks source link

Building VirtualBox kernel modules with Clang and DKMS #1104

Closed torvic9 closed 2 years ago

torvic9 commented 4 years ago

Hi,

I don't know where to report this, but...let me try :) feel free to close if it's the wrong place.

So I finally managed to build current 5.7.10 kernel with LLVM=1 LLVM_IAS=1 with clang11 from the release/11.x branch, and the IAS related patches I found (among others) here: https://github.com/ClangBuiltLinux/linux/issues/1049 , and which I assembled together here: https://github.com/torvic9/linux57-vd/blob/master/0006-clang-llvm-ias-support.patch

The problem is that I cannot build the VBox host modules using CC=clang LD=ld.lld. I get the following errors:

./arch/x86/include/asm/page_64.h:49:2: error: expected '(' after 'asm'
        alternative_call_2(clear_page_orig,
        ^
/arch/x86/include/asm/segment.h:266:2: error: expected '(' after 'asm'
        alternative_io ("lsl %[seg],%[p]",
        ^

./arch/x86/include/asm/special_insns.h:212:2: error: expected '(' after 'asm'
        alternative_io(".byte " __stringify(NOP_DS_PREFIX) "; clflush %P0",
        ^

and many more of the same type.

Compiling the kernel with clang10 and without LLVM_IAS results in working VBox modules.

Is this an issue with the fact that I used LLVM_IAS for the kernel? Or is it rather related to using clang11 instead of clang10?

Thanks!

dileks commented 4 years ago

LLVM_IAS=1 is still categorized EXPERIMENTAL.

Unfortunately, you forgot about some important informations which might help. What (snapshot) versions of clang-10 and clang-11? What version of virtualbox?

As said in #1049: Linux v5.8-rcX together with LLVM_IAS=1 and clang-11 is a good choice. Doing some -no-integrated-as workarounds helped but not in any case with Linux v5.7.y. As you have seen by yourself it depends also on the clang version.

Compiling/assembling LLVM_IAS=1 plus LLVM=1 is another story.

Currently, I use this kernel-release:

$ cat /proc/version 
Linux version 5.8.0-rc6-4-amd64-llvm11-ias (sedat.dilek@gmail.com@iniza) (clang version 11.0.0 (https://github.com/llvm/llvm-project cebd637c88624dfd44520848cb1f43dc8a02ba80), LLD 11.0.0 (https://github.com/llvm/llvm-project cebd637c88624dfd44520848cb1f43dc8a02ba80)) #4~bullseye+dileks1 SMP 2020-07-22

...with some additional patches - the one to support LLVM_IAS=1 are in my series: 5 patches.

Unsure - when I will update #1049 - but Linux v5.7.y is no more on my radar or interest. I have Debian's 5.7.6 plus my Linux 5.8-rcX (LLVM_IAS=1 + LLVM=1) kernel installed.

Parts like "elfcore: build with -no-integrated-as when LLVM_IAS=1" I published in #1049. Some parts (diffs) I do not know where you have it from and what it is fixing.

Without further informations you are at your own...

Update: Add clang-11 to good choice.

nickdesaulniers commented 4 years ago

The problem is that I cannot build the VBox host modules

What config is that?

dileks commented 4 years ago

Updated #1049.

[1] https://github.com/ClangBuiltLinux/linux/issues/1049#issuecomment-662609378

torvic9 commented 4 years ago

Here is the version info: clang version 11.0.0 (git://github.com/llvm/llvm-project 8f8ec9927623eb54f003993b3d8411fc0130ca90)

I'm not sure it's the IAS. The modules cannot be built either way with the clang version mentioned above, i.e. with the kernel built with/without IAS. The kernel itself builds, installs and runs just fine.

Is it possible thatt additional flags have to be passed to the VBox make command?

What works is clang10.0.1(rc4) without IAS, on the same kernel+config.

ihalip commented 4 years ago

What config is that?

VirtualBox modules are out-of-tree.

dileks commented 4 years ago

We have parts of "VBOX" in-tree - depending on your Linux-kernel version :-) - not a full support like VirtualBox upstream.

$ grep VBOX /boot/config-5.8.0-rc6-4-amd64-llvm11-ias 
CONFIG_DRM_VBOXVIDEO=m
CONFIG_VBOXGUEST=m
CONFIG_VBOXSF_FS=m

Here on Debian we have virtualbox-dkms to build out-of-tree with DKMS.

torvic9 commented 4 years ago

Yes, I mean the out-of-tree host modules (vboxdrv etc.). I also use dkms to build them, with the following make command added to dkms.conf:

MAKE[0]='make CC=clang LD=ld.lld KERNELRELEASE=${_kernver}' (_kernver is of course the kernel version)

dileks commented 4 years ago

I am building a new kernel and will try with virtualbox-dkms version 6.1.12-dfsg-6 from Debian/unstable after it has finished.

torvic9 commented 4 years ago

Thanks! You're right, I forgot some info. VBox version is 6.1.12. clang10 is: clang version 10.0.1 (git://github.com/llvm/llvm-project 77d76b71d7df39b573dfa1e391096a040e9b7bd3) clang11 is: clang version 11.0.0 (git://github.com/llvm/llvm-project 8f8ec9927623eb54f003993b3d8411fc0130ca90)

Some parts (diffs) I do not know where you have it from and what it is fixing.

The only part that I didn't get from here is the last diff, which eliminates __force_order errors I encountered. Here's the link to LKML: https://lore.kernel.org/r/20200527135329.1172644-1-arnd@arndb.de/

I'm aware that IAS is still experimental, I'm just trying out a few things (and, as you can see, I don't have the required knowledge...). I have not yet moved to 5.8.

I'm typing this from 5.7.10 built with LLVM=1 LLVM_IAS=1: Linux version 5.7.10-1-vd (linux57-vd@manjaro) (clang-vd clang version 11.0.0 (git://github.com/llvm/llvm-project 8f8ec9927623eb54f003993b3d8411fc0130ca90)

I can paste the logfile from the failed module building process if that helps.

nickdesaulniers commented 4 years ago

Ah! out of tree/dkms. Yeah, that's not tested. IIRC, @ColinIanKing mentioned being able to work around this. @ColinIanKing do you remember the particular issue you saw with DKMS?

@torvic9 how do we fetch this out of tree code, and invoke a build of it?

dileks commented 4 years ago

You need linux-headers packages installed:

# dpkg -l | grep linux-headers | awk '/^ii/ {print $1 " " $2 " " $3}' | column -t
ii  linux-headers-5.7.0-1-amd64   5.7.6-1
ii  linux-headers-5.7.0-1-common  5.7.6-1
ii  linux-headers-amd64           5.7.6-1

Install VBox DKMS package, build+install and check DKMS status:

# apt update && apt install virtualbox-dkms

# dpkg -l | grep dkms | awk '/^ii/ {print $1 " " $2 " " $3}' | column -t
ii  acpi-call-dkms   1.1.0-6
ii  dkms             2.8.2-2
ii  virtualbox-dkms  6.1.12-dfsg-6

# dkms install -m virtualbox -v 6.1.12 -k 5.7.0-1-amd64

# dkms status
acpi-call, 1.1.0, 5.7.0-1-amd64, x86_64: installed
virtualbox, 6.1.12, 5.7.0-1-amd64, x86_64: installed
dileks commented 4 years ago 
$ sudo dpkg -i linux-headers-5.8.0-rc6-6-amd64-llvm11-ias_5.8.0~rc6-6~bullseye+dileks1_amd64.deb

$ sudo dkms build -m virtualbox -v 6.1.12 -k 5.8.0-rc6-6-amd64-llvm11-ias

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
make -j4 KERNELRELEASE=5.8.0-rc6-6-amd64-llvm11-ias -C /lib/modules/5.8.0-rc6-6-amd64-llvm11-ias/build M=/var/lib/dkms/virtualbox/6.1.12/build....(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.8.0-rc6-6-amd64-llvm11-ias (x86_64)
Consult /var/lib/dkms/virtualbox/6.1.12/build/make.log for more information.

make-log_vbox-6-1-12_llvm11-ias.txt

dileks commented 4 years ago

My hack was to add a temporary symlink /path/to/clang-$VER -> /usr/bin/gcc.

dileks commented 4 years ago

With that hack:

# cd /usr/bin/
# mv gcc gcc.orig
# ln -sf clang-11 gcc

# dkms build -m virtualbox -v 6.1.12 -k 5.8.0-rc6-6-amd64-llvm11-ias            

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
make -j4 KERNELRELEASE=5.8.0-rc6-6-amd64-llvm11-ias -C /lib/modules/5.8.0-rc6-6-amd64-llvm11-ias/build M=/var/lib/dkms/virtualbox/6.1.12/build......(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.8.0-rc6-6-amd64-llvm11-ias (x86_64)
Consult /var/lib/dkms/virtualbox/6.1.12/build/make.log for more information.

# cat /var/lib/dkms/virtualbox/6.1.12/build/make.log
DKMS make.log for virtualbox-6.1.12 for kernel 5.8.0-rc6-6-amd64-llvm11-ias (x86_64)
Thu Jul 23 19:36:14 CEST 2020
make: Entering directory '/usr/src/linux-headers-5.8.0-rc6-6-amd64-llvm11-ias'
  CC [M]  /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.o
  CC [M]  /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/SUPDrv.o
  CC [M]  /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/SUPDrvGip.o
  CC [M]  /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/SUPDrvSem.o
  CC [M]  /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/SUPDrvTracer.o
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
    RTCCUINTREG uOld = this_cpu_read(cpu_tlbstate.cr4);
                                     ^
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:38: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:760:17: error: initializing 'RTCCUINTREG' (aka 'unsigned long long') with an expression of incompatible type 'void'
    RTCCUINTREG uOld = this_cpu_read(cpu_tlbstate.cr4);
                ^      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
        this_cpu_write(cpu_tlbstate.cr4, uNew);
                       ^
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.c:764:24: error: use of undeclared identifier 'cpu_tlbstate'
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[2]: *** [scripts/Makefile.build:281: /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/linux/SUPDrv-linux.o] Fehler 1
make[2]: *** Es wird auf noch nicht beendete Prozesse gewartet....
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/SUPDrvTracer.o: warning: objtool: .text+0x7: indirect jump found in RETPOLINE build
/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/SUPDrvTracer.o: warning: objtool: supdrvTracerProbeFireStub() is missing an ELF size annotation
make[1]: *** [scripts/Makefile.build:497: /var/lib/dkms/virtualbox/6.1.12/build/vboxdrv] Fehler 2
make: *** [Makefile:1762: /var/lib/dkms/virtualbox/6.1.12/build] Error 2
make: Leaving directory '/usr/src/linux-headers-5.8.0-rc6-6-amd64-llvm11-ias'
dileks commented 4 years ago

The problem is in DKMS code:

3614: # Unset environment variables that may interfere with the build
3615: unset CC CXX CFLAGS CXXFLAGS LDFLAGS

[1] https://github.com/dell/dkms/blob/master/dkms#L3614

torvic9 commented 4 years ago

My hack was to add a temporary symlink /path/to/clang-$VER -> /usr/bin/gcc.

I worked around it by adding the make command with CC=clang to dkms.conf (see above).

Edit: however your error messages are completely different to the asm errors?

dileks commented 4 years ago

@torvic9 ah cool.

Hmm, you need to add all $MAKE_OPTS from your make line of the kernel-build including LLVM_IAS=1 and LLVM=1?

dileks commented 4 years ago

Edit: however your error messages are completely different to the asm errors?

I have some additional x86/asm patches in my series and Linux v5.8-rc6. Cannot say which VBox version supports it.

torvic9 commented 4 years ago

I only added CC. I'm now rebuilding clang11 and will try again, however still on 5.7. Will switch to 5.8 soon though.

Where can I find your current series?

dileks commented 4 years ago

That's the minimal patchset:

https://github.com/ClangBuiltLinux/linux/issues/1049#issuecomment-662609378

torvic9 commented 4 years ago

(I deleted my previous comment as it was just a typo)

dileks commented 4 years ago

There are two different version of @arndb crc32c patch - the original [1] is for Linux 5.8 - I published an adapted version for Linux v5.7 in #1043.

[1] https://git.kernel.org/herbert/cryptodev-2.6/c/44623b2818f4a442726639572f44fd9b6d0ef68c

dileks commented 4 years ago

Depending on your llvm-toolchain (here: release/11.x Git) - the patches you mentioned are no more necessary. In my last posting in #1043 I displayed the version I used.

torvic9 commented 4 years ago

Depending on your llvm-toolchain (here: release/11.x Git) - the patches you mentioned are no more necessary.

Here: git-release/11.x from July 21st. Still necessary. I haven't updated the toolchain yet due to a problem with the build-llvm.py script (I'll open an issue there).

To get back on topic: I still encounter the same ASM related errors on 5.8-rc6 when compiling VBox host modules 6.1.12.

dileks commented 4 years ago

Nice to see you use our tools like tc-build and infrastructure :-).

As said https://github.com/llvm/llvm-project/commit/cebd637c88624dfd44520848cb1f43dc8a02ba80 fixed for me all recent Linux/x86 problems reported in the CBL issue-tracker. So, I will keep this version for a while. Maybe till there is an official llvmorg-11.0.0-rc1 release tag.

[1] https://github.com/ClangBuiltLinux/tc-build/issues/104 [2] https://github.com/llvm/llvm-project/tags

dileks commented 4 years ago

Depending on your llvm-toolchain (here: release/11.x Git) - the patches you mentioned are no more necessary.

Here: git-release/11.x from July 21st. Still necessary.

Be precise as possible via clang -v. The output displays a unique commit-id when build from Git.

dileks commented 4 years ago

You looked at https://github.com/llvm/llvm-project/commit/833f8c958601bb640ba6a25d627c1dc58dad14d2?

torvic9 commented 4 years ago

Be precise as possible via clang -v.

I did, maybe you missed it: clang version 11.0.0 (git://github.com/llvm/llvm-project 8f8ec9927623eb54f003993b3d8411fc0130ca90)

I'll build a new toolchain later and report back; this laptop doesn't have enough processing power for a PGO build...

Nice to see you use our tools like tc-build and infrastructure :-).

Sure, it's fun experimenting with clang :) I hope I can get a better knowledge about it. Technically I don't have any real use for it.

dileks commented 4 years ago

Hehe, it's CPU power plus enough RAM (and/or SWAP) for PGO and/or Thin-LTO.

BTW, I am doing a stage1-only build.

nickdesaulniers commented 4 years ago

ok, let's report then to the DKMS developers. Let's find their bug tracker.

dileks commented 4 years ago

@nickdesaulniers

This is a known issue which I reported as "dkms: Build with an alternative like compiler rather than gcc" to debian-kernel ML in July 2018.

Mario Limonciello pointed me to the above lines in the dkms source.

It does not hurt to report to dkms upstream but as @torvic9 pointed out you can override/pass individual make options via/to dkms.conf. ( Untested by /me. )

[1] https://lists.debian.org/debian-kernel/2018/07/subject.html

torvic9 commented 4 years ago

The issue is not related to LLVM_IAS=1.

With 5.8-rc6 and clang-11 3c1fca803bc14617b67ba2125e1b4b77190e9f86, I get the same ASM errors with and without IAS.

However, using instead clang-10 77d76b71d7df39b573dfa1e391096a040e9b7bd3, I do not get these ASM errors, instead I have similar error messages as in @dileks comment above.

It seems that the host modules are not yet fully supported, a similar bug report can be found here: https://bugs.launchpad.net/ubuntu/+source/virtualbox/+bug/1884652 (see log file)

torvic9 commented 4 years ago

Here are some patches to make VBox 6.1.12 host modules compatible with kernel 5.8. I have only tried with gcc so far, but they seem to build correctly. There is one patch for the kernel itself, and 3 patches for the host modules.

https://www.virtualbox.org/ticket/19644

dileks commented 4 years ago

That means be patient - wait for VBox version 6.1.14 which will have support for Linux-kernel version 5.8.

UPDATE: s/6.1.13/6.1.14 (stable versions have even numbers)

dileks commented 4 years ago

[1] says:

virtualbox (6.1.12-dfsg-8) unstable; urgency=medium

  * debian/patches/kernel-5.8-1.patch:
  * debian/patches/kernel-5.8-2.patch:
  * debian/patches/kernel-5.8-3.patch:
    - apply upstream proposed patches to fix a build failure with kernel 5.8
      (note, this requires also a kernel patch)
      https://www.virtualbox.org/ticket/19644

 -- Gianfranco Costamagna <locutusofborg@debian.org>  Thu, 30 Jul 2020 18:58:32 +0200

[1] https://metadata.ftp-master.debian.org/changelogs//contrib/v/virtualbox/virtualbox_6.1.12-dfsg-8_changelog

dileks commented 4 years ago

Missing patch against Linux v5.8-rc7:

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -315,6 +315,7 @@ int map_kernel_range_noflush(unsigned long addr, unsigned long size,

        return 0;
 }
+EXPORT_SYMBOL(map_kernel_range);

 int map_kernel_range(unsigned long start, unsigned long size, pgprot_t prot,
                struct page **pages)
@@ -2129,6 +2130,7 @@ struct vm_struct *__get_vm_area_caller(unsigned long size, unsigned long flags,
        return __get_vm_area_node(size, 1, flags, start, end, NUMA_NO_NODE,
                                  GFP_KERNEL, caller);
 }
+EXPORT_SYMBOL(__get_vm_area_caller);

 /**
  * get_vm_area - reserve a contiguous kernel virtual area

UPDATE: Attached a simple RFC patch without any Links or Credits.

0001-mm-vmalloc-Export-map_kernel_range-and-__get_vm_area_patch.txt

0001-mm-vmalloc-vbox-Export-map_kernel_range-and-__get_vm_patch.txt

dileks commented 4 years ago

VBox patches to build Linux v5.8-rcX:

  1. https://sources.debian.org/src/virtualbox/6.1.12-dfsg-8/debian/patches/
  2. https://sources.debian.org/data/contrib/v/virtualbox/6.1.12-dfsg-8/debian/patches/kernel-5.8-1.patch
  3. https://sources.debian.org/data/contrib/v/virtualbox/6.1.12-dfsg-8/debian/patches/kernel-5.8-2.patch
  4. https://sources.debian.org/data/contrib/v/virtualbox/6.1.12-dfsg-8/debian/patches/kernel-5.8-3.patch
torvic9 commented 4 years ago

Thanks. Already applied all 4 directly from vbox site, i.e. put them in PKGBUILDs (vbox and kernel). No missing patch there. Builds fine with rc7 and gcc. Clang not tested yet.

(Rather wait for .14, I think odd numbers are dev versions)

dileks commented 4 years ago

@torvic9

Good. I will try with the kernel-patch only as Debian/unstable has the 3 other ones. Here it's 36 degrees Celsius @18:02 so I will do build this night.

torvic9 commented 4 years ago

35° here :) good luck. I'll try with clang as well and report back.

dileks commented 4 years ago

I was able to build a Vbox kernel-modules with my gcc-symlink trick. Need some more playing with dkms.conf.

dileks commented 4 years ago

Passing LLVM_IAS=1 results in:

  AR      /var/lib/dkms/virtualbox/6.1.12/build/built-in.a
  MODPOST /var/lib/dkms/virtualbox/6.1.12/build/Module.symvers
ERROR: modpost: "__force_order" [/var/lib/dkms/virtualbox/6.1.12/build/vboxdrv/vboxdrv.ko] undefined!

Patch from @arndb https://lkml.org/lkml/2020/5/27/766

[1] https://lore.kernel.org/patchwork/patch/1248382/

dileks commented 4 years ago

Applied @arndb patch on top of Linux v5.8-rc7.

With this diff:

--- /usr/src/virtualbox-6.1.12/dkms.conf.orig   2020-07-30 18:58:32.000000000 +0200
+++ /usr/src/virtualbox-6.1.12/dkms.conf        2020-08-01 11:57:36.420490455 +0200
@@ -1,5 +1,6 @@
 PACKAGE_NAME="virtualbox"
 PACKAGE_VERSION="6.1.12"
+MAKE="make CC=clang-11 LD=ld.lld-11 AR=llvm-ar-11 NM=llvm-nm-11 OBJCOPY=llvm-objcopy-11 OBJDUMP=llvm-objdump-11 OBJSIZE=llvm-size-11 READELF=llvm-readelf-11 STRIP=llvm-strip-11 LLVM_IAS=1 -C /lib/modules/5.8.0-rc7-6-amd64-llvm11-ias/build M=/var/lib/dkms/virtualbox/6.1.12/build"
 CLEAN="rm -f *.*o"
 BUILT_MODULE_NAME[0]="vboxdrv"
 BUILT_MODULE_LOCATION[0]="vboxdrv"

Build VBox kernel-modules:

root@iniza:~# LC_ALL=C dkms build -m virtualbox -v 6.1.12 -k 5.8.0-rc7-6-amd64-llvm11-ias              

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
make -j4 KERNELRELEASE=5.8.0-rc7-6-amd64-llvm11-ias CC=clang-11 LD=ld.lld-11 AR=llvm-ar-11 NM=llvm-nm-11 OBJCOPY=llvm-objcopy-11 OBJDUMP=llvm-objdump-11 OBJSIZE=llvm-size-11 READELF=llvm-readelf-11 STRIP=llvm-strip-11 LLVM_IAS=1 -C /lib/modules/5.8.0-rc7-6-amd64-llvm11-ias/build M=/var/lib/dkms/virtualbox/6.1.12/build.........................
cleaning build area...

DKMS: build completed.

Install VBox kernel-modules:

root@iniza:~# LC_ALL=C dkms install -m virtualbox -v 6.1.12 -k 5.8.0-rc7-6-amd64-llvm11-ias

Check DKMS status:

root@iniza:~# dkms status -m virtualbox
virtualbox, 6.1.12, 5.8.0-rc7-6-amd64-llvm11-ias, x86_64: installed

Check status of virtualbox.service and start it:

root@iniza:~# systemctl status virtualbox.service 
● virtualbox.service - LSB: VirtualBox Linux kernel module
     Loaded: loaded (/etc/init.d/virtualbox; generated)
     Active: inactive (dead)
       Docs: man:systemd-sysv-generator(8)

root@iniza:~# systemctl start virtualbox.service 

root@iniza:~# systemctl status virtualbox.service 
● virtualbox.service - LSB: VirtualBox Linux kernel module
     Loaded: loaded (/etc/init.d/virtualbox; generated)
     Active: active (exited) since Sat 2020-08-01 12:00:41 CEST; 8min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 8247 ExecStart=/etc/init.d/virtualbox start (code=exited, status=0/SUCCESS)

Aug 01 12:00:41 iniza systemd[1]: Starting LSB: VirtualBox Linux kernel module...
Aug 01 12:00:41 iniza virtualbox[8247]: Loading VirtualBox kernel modules... vboxdrv vboxnetflt vboxnetadp.
Aug 01 12:00:41 iniza systemd[1]: Started LSB: VirtualBox Linux kernel module.

Check loaded VBox kernel-modules:

root@iniza:~# lsmod | grep vbox
vboxnetadp             28672  0
vboxnetflt             32768  0
vboxdrv               520192  2 vboxnetadp,vboxnetflt

Check dmesg-outputs:

root@iniza:~# LC_ALL=C dmesg -T | tail
[Sat Aug  1 11:47:13 2020] wlp1s0: associate with 50:d4:f7:2e:17:da (try 1/3)
[Sat Aug  1 11:47:13 2020] wlp1s0: RX AssocResp from 50:d4:f7:2e:17:da (capab=0x411 status=0 aid=1)
[Sat Aug  1 11:47:13 2020] wlp1s0: associated
[Sat Aug  1 11:47:13 2020] IPv6: ADDRCONF(NETDEV_CHANGE): wlp1s0: link becomes ready
[Sat Aug  1 12:00:40 2020] vboxdrv: loading out-of-tree module taints kernel.
[Sat Aug  1 12:00:40 2020] vboxdrv: Found 4 processor cores
[Sat Aug  1 12:00:40 2020] vboxdrv: TSC mode is Invariant, tentative frequency 1597812133 Hz
[Sat Aug  1 12:00:40 2020] vboxdrv: Successfully loaded version 6.1.12_Debian (interface 0x002d0001)
[Sat Aug  1 12:00:40 2020] VBoxNetFlt: Successfully started.
[Sat Aug  1 12:00:40 2020] VBoxNetAdp: Successfully started.
dileks commented 4 years ago

New VBox dkms.conf diff:

--- /usr/src/virtualbox-6.1.12/dkms.conf.orig   2020-07-30 18:58:32.000000000 +0200
+++ /usr/src/virtualbox-6.1.12/dkms.conf        2020-08-01 13:06:54.522773507 +0200
@@ -1,6 +1,11 @@
 PACKAGE_NAME="virtualbox"
 PACKAGE_VERSION="6.1.12"
-CLEAN="rm -f *.*o"
+CC_FOR_BUILD="CC=clang-11"
+LD_FOR_BUILD="LD=ld.lld-11"
+LLVM_UTILS="AR=llvm-ar-11 NM=llvm-nm-11 OBJCOPY=llvm-objcopy-11 OBJDUMP=llvm-objdump-11 OBJSIZE=llvm-size-11 READELF=llvm-readelf-11 STRIP=llvm-strip-11"
+MAKE_OPTS="${CC_FOR_BUILD} ${LD_FOR_BUILD} ${LLVM_UTILS} LLVM_IAS=1"
+MAKE[0]="make V=1 ${MAKE_OPTS} -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build"
+CLEAN="make -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build clean"
 BUILT_MODULE_NAME[0]="vboxdrv"
 BUILT_MODULE_LOCATION[0]="vboxdrv"
 DEST_MODULE_LOCATION[0]="/updates"
dileks commented 4 years ago

Full VBox dkms.conf:

[ /usr/src/virtualbox-6.1.12/dkms.conf ]

PACKAGE_NAME="virtualbox"
PACKAGE_VERSION="6.1.12"
CC_FOR_BUILD="CC=clang-11"
LD_FOR_BUILD="LD=ld.lld-11"
LLVM_UTILS="AR=llvm-ar-11 NM=llvm-nm-11 OBJCOPY=llvm-objcopy-11 OBJDUMP=llvm-objdump-11 OBJSIZE=llvm-size-11 READELF=llvm-readelf-11 STRIP=llvm-strip-11"
MAKE_OPTS="${CC_FOR_BUILD} ${LD_FOR_BUILD} ${LLVM_UTILS} LLVM_IAS=1"
MAKE[0]="make V=1 ${MAKE_OPTS} -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build"
CLEAN="make -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build clean"
BUILT_MODULE_NAME[0]="vboxdrv"
BUILT_MODULE_LOCATION[0]="vboxdrv"
DEST_MODULE_LOCATION[0]="/updates"
BUILT_MODULE_NAME[1]="vboxnetadp"
BUILT_MODULE_LOCATION[1]="vboxnetadp"
DEST_MODULE_LOCATION[1]="/updates"
BUILT_MODULE_NAME[2]="vboxnetflt"
BUILT_MODULE_LOCATION[2]="vboxnetflt"
DEST_MODULE_LOCATION[2]="/updates"
AUTOINSTALL="yes"
dileks commented 4 years ago

I was able to import my stretch-buildd.vmdk, boot VM and shutdown.

torvic9 commented 4 years ago

No luck here. Still the same error messages as in OP:

In file included from ./arch/x86/include/asm/page.h:12:
./arch/x86/include/asm/page_64.h:49:2: error: expected '(' after 'asm'
        alternative_call_2(clear_page_orig,
        ^

Any ideas?

Edit: happens both with and without IAS-built kernel.

dileks commented 4 years ago

@torvic9 I do not see this. But I have some extra patches applied like from <tip.git#x86/asm>.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/asm

dileks commented 4 years ago

I have sent an email to the Debian virtualbox maintainers and asked for an alternative handling of dkms.conf within virtualbox-dkms package in Debian/unstable.

torvic9 commented 4 years ago

I'm giving up on this one, it's not a big issue for me as I can just build with either clang-10 or gcc. I retried with binutils 2.35, x86/asm from tip as suggested, and a freshly built toolchain, but no... still the same ASM error messages, happens with every version of clang-11 I've tried. Could be a config issue, but I'm not going to investigate further anymore, already spent too much time with this ;)