vmware-modules fails to emerge with gentoo-sources-5.4.21

[realnc]

I just upgraded my kernel from 4.19 to 5.4 (new LTS). Emerging vmware-modules-15.5.1-r1 fails with:

>>> Emerging (1 of 1) app-emulation/vmware-modules-15.5.1-r1::local
 * vmware-modules-15.5.1-5.4.zip BLAKE2B SHA512 size ;-) ...                                                                                                  [ ok ]
 * Determining the location of the kernel source code
 * Found kernel source directory:
 *     /usr/src/linux
 * Found sources for kernel version:
 *     5.4.21-gentoo
 * Checking for suitable kernel configuration options...                                                                                                      [ ok ]
 * Checking for suitable kernel configuration options...                                                                                                      [ ok ]
>>> Unpacking source...
>>> Unpacking vmware-modules-15.5.1-5.4.zip to /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work
>>> Source unpacked in /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work
>>> Preparing source in /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4 ...
 * Preparing vmmon module
make -j4 HOSTCC=x86_64-pc-linux-gnu-gcc CROSS_COMPILE=x86_64-pc-linux-gnu- 'LDFLAGS=-m elf_x86_64' auto-build KERNEL_DIR=/usr/src/linux KBUILD_OUTPUT=/usr/src/linux 
Using kernel build system.
make -C /usr/src/linux/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
  MODULEBUILDDIR= modules
make[1]: Entering directory '/usr/src/linux-5.4.21-gentoo'
 * ACCESS DENIED:  fopen_wr:     /usr/src/linux-5.4.21-gentoo/.63.tmp
 * ACCESS DENIED:  unlinkat:     /usr/src/linux-5.4.21-gentoo/.63.tmp
rm: cannot remove '.63.tmp': Permission denied
 * ACCESS DENIED:  unlinkat:     /usr/src/linux-5.4.21-gentoo/.63.o
rm: cannot remove '.63.o': Permission denied
 * ACCESS DENIED:  fopen_wr:     /usr/src/linux-5.4.21-gentoo/.68.tmp
 * ACCESS DENIED:  unlinkat:     /usr/src/linux-5.4.21-gentoo/.68.tmp

There's a lot more "ACCESS DENIED" errors. I tried to emerge with:

FEATURES="-sandbox -usersandbox"

but this results in a different error:

>>> Compiling source in /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4 ...
 * Preparing vmmon module
make -j4 HOSTCC=x86_64-pc-linux-gnu-gcc CROSS_COMPILE=x86_64-pc-linux-gnu- 'LDFLAGS=-m elf_x86_64' auto-build KERNEL_DIR=/usr/src/linux KBUILD_OUTPUT=/usr/src/linux 
Using kernel build system.
make -C /usr/src/linux/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
  MODULEBUILDDIR= modules
make[1]: Entering directory '/usr/src/linux-5.4.21-gentoo'
  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c:410:1: fatal error: opening dependency file scripts/basic/.fixdep.d: Permission denied

[stefantalpalaru]

Did you build a kernel in "/usr/src/linux"? Or at least did a make config?

That whole fixdep part is not supposed to kick in at all. The process looks like this on my system:

>>> Compiling source in /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4 ...
 * Preparing vmmon module
make -j9 -l9 HOSTCC=x86_64-pc-linux-gnu-gcc CROSS_COMPILE=x86_64-pc-linux-gnu- 'LDFLAGS=-m elf_x86_64' auto-build KERNEL_DIR=/usr/src/linux KBUILD_OUTPUT=/usr/src/linux 
Using kernel build system.
make -C /usr/src/linux/include/.. M=$PWD SRCROOT=$PWD/. \
  MODULEBUILDDIR= modules
make[1]: Entering directory '/usr/src/linux-5.4.11-gentoo'
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/linux/driverLog.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/linux/driver.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/linux/hostif.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/common/memtrack.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/common/apic.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/common/statVarsVmmon.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/common/vmx86.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/common/sharedAreaVmmon.o
  CC [M]  /var/tmp/portage/app-emulation/vmware-modules-15.5.1-r1/work/vmware-host-modules-w15.5.1-k5.4/vmmon-only/common/cpuid.o

[realnc]

Yes, I did build the kernel and also did make install & make modules_install. And obviously eselect kernel set 2 prior to that.

However, I now know what's going on. You can't build vmware-modules while the new kernel is not booted. Normally,I build the new kernel, to the install and module_install, I then emerge @module-rebuild and reboot. This always worked fine when upgrading from 4.19.x to 4.19.y. But with the new kernel being 5.4 while the running kernel is 4.19, it doesn't work in the case of vmware-modules. After I booted the 5.4 kernel, then emerging vmware-modules worked.

So we can close this, I guess, unless there's something that can be done about it in the ebuild?

[stefantalpalaru]

You can't build vmware-modules while the new kernel is not booted.

That's unusual.

Normally,I build the new kernel, to the install and module_install, I then emerge @module-rebuild and reboot.

That's what I also do, and I never ran into this problem.

unless there's something that can be done about it in the ebuild?

I don't see what we can do in there, without understanding the root cause. Closing the issue, but feel free to comment if you discover some other clue.

[cfuga]

Yay! I found the answer.

This bug appears when you're compiling vmware-modules for a 5.x kernel while you're still running a 4.x kernel (or lower).

The Makefiles in vmmon-only and vmnet-only subdirs define $VM_UNAME as

VM_UNAME = $(shell uname -r)

$VM_UNAME is used to select M= for 5.x kernels, instead of SUBDIRS=

# Use SUBDIRS on 2.x, 3.x, 4.x.  Use M on newer kernels.
ifeq ($(filter-out 2 3 4,$(firstword $(subst ., ,$(VM_UNAME)))),)
DIRVAR := SUBDIRS
else
DIRVAR := M
endif

When the running kernel version is 4.x or lower, the kernel build system uses the wrong variable, and the sandbox errors appear.

The following patch for the latest available ebuild fixes the bug.

--- vmware-modules-16.0.0.ebuild   2020-09-22 06:00:13.189601642 -0500
+++ vmware-modules-16.0.0.ebuild        2020-10-12 22:50:48.296361930 -0500
@@ -52,6 +52,7 @@
 src_prepare() {
        # decouple the kernel include dir from the running kernel version: https://github.com/stefantalpalaru/gentoo-overlay/issues/17
        sed -i -e "s%HEADER_DIR = /lib/modules/\$(VM_UNAME)/build/include%HEADER_DIR = ${KERNEL_DIR}/include%" */Makefile || die "sed failed"
+       sed -i -e "s%VM_UNAME = \$(shell uname -r)%VM_UNAME = ${KV_FULL}%" */Makefile || die "sed failed"

        # Allow user patches so they can support RC kernels and whatever else
        default

[stefantalpalaru]

Beautiful! Fix applied in 15.5.6-r3 and 16.0.0-r1.