openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.45k stars 1.73k forks source link

ZFS 2.1.5 Cannot build kernel modules against the linux kernel 5.4.17 #13716

Open vovanoti opened 2 years ago

vovanoti commented 2 years ago

System information

Distribution Name | Oracle -- | -- Distribution Version | 8.4 Kernel Version | 5.4.17 Architecture | x86_64 OpenZFS Version | 2.1.5

Steps To Reproduce

Steps to reproduce the behavior:

  1. sh autogen.sh
  2. ./configure --with-spec=redhat --with-linux=/usr/src/kernels/5.4.17-2102.203.6.el8uek.x86_64 --with-linux-obj=/usr/src/kernels/5.4.17-2102.203.6.el8uek.x86_64
  3. make rpm-kmod

Build log

...
checking kernel source directory... /usr/src/kernels/4.18.0-305.17.1.el8_4.x86_64
checking kernel build directory... /usr/src/kernels/4.18.0-305.17.1.el8_4.x86_64
...

Additional context

I'm trying to build kmod packages against the kernel 5.4.17. I specified a kernel in a non-default location with flags --with-linux and --with-linux-obj (as described in the offical manual) Building process passed succesfully but kernel modules installing in the old kernel directory. I have kernel named 5.4.17-2102.203.6.el8uek.x8664 and have installed necessary dependencies (such as kernel-uek-devel-5.4.17). I also tried to delete the kernel-devel package for the old kernel (kernel-devel-4.18.0) or make a symlink from /usr/src/kernels/5.4.17-2102.203.6.el8uek.x8664/include/linux to _/usr/src/kernels/4.18.0-305.17.1.el8_4.x8664/include/linux but I got errors during the building and nothing is going to help. Below you can see the output one of the builded kmod packages (and they all (_kmod-zfs-2.1.5-1.el8.x86_64.rpm kmod-zfs-debuginfo-2.1.5-1.el8.x86_64.rpm kmod-zfs-devel-2.1.5-1.el8.x8664.rpm) get the old kernel version in the path). All it looks like ./configure starts after make rpm-kmod and change my specifed kernel to the default. BTW the zfs-dkms-2.1.5 package is building and working fine but I have some issues with it when trying to install lustre-zfs-dkms-2.15.0 so, if it is possible, I want to build zfs kernel module against the kernel 5.4.17. Client zfs packages are fine too.

Kindly asking you to help me to build kmod packages against the specified kernel 5.4.17.

[root@K1EYVO zfs]# rpm -ql kmod-zfs-2.1.5-1.el8.x86_64.rpm
/lib/modules/4.18.0-305.17.1.el8_4.x86_64
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/avl
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/avl/zavl.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/icp
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/icp/icp.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/lua
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/lua/zlua.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/nvpair
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/nvpair/znvpair.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/spl
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/spl/spl.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/unicode
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/unicode/zunicode.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/zcommon
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/zcommon/zcommon.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/zfs
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/zfs/zfs.ko
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/zstd
/lib/modules/4.18.0-305.17.1.el8_4.x86_64/extra/zfs/zstd/zzstd.ko
...

config.log

rincebrain commented 2 years ago

5.4.17 and "whatever Oracle patched in" are not at all the same version, so please mention the full version number when discussing this.

I think the last person who tried to get UEK kmods working gave up.

behlendorf commented 2 years ago

Duplicate of #13625. The UEK environment is sufficiently different from RHEL / Alma / Rocky Linux that the rpm packaging in the repository is not compatible. If someone familiar with the UEK differences would like to work on extending the packaging that would be welcome.

mightyspiky commented 2 years ago

I've compiled kABI-tracking kmod successfully for UEK R6 (5.4.17) and UEK R7 (5.15.0) on x86_64, thanks to the help of rincebrain and behlendorf. For UEK R6, you should only have to replace the string "kernel-devel" by "kernel-uek-devel" in /usr/lib/rpm/macros.d/macros.kmp (absolute path) and rpm/generic/zfs-kmod.spec.in (relative to the untared zfs source archive). After that, be sure to have kernel-uek-devel rpm installed and it should build correctly. Please note though that modifying /usr/lib/rpm/macros.d/macros.kmp is not very clean because it is owned by the kernel-rpm-macros rpm and it will be overwritten after updating that package.

mightyspiky commented 2 years ago

When compiling for a different kernel than the one currently booted, please note that the "--with-linux" and "--with-linux-obj" flags didn't work for me. I had to do a trick with the uname command as described in https://github.com/openzfs/zfs/issues/13625#issuecomment-1182596279 or else the compilation would fail while trying searching for the currently booted kernel headers in /usr/src/kernels

vovanoti commented 2 years ago

Thank you very much for your answers. We don't need to build ZFS right now so if it will be necessary I will return to the @mightyspiky advice. Thank you all once again :)

crazyhouse33 commented 2 years ago

I have an issue that seem to also indicate that there is an issue with "with-linux" and "--with-linux-obj" on 5.15.63, zfs 2.1.5, but since I am using grsec, I will just put it there as there is the possibility it's grsec related.

When building zfs like this:

cd $ZFS_BUILD
    sh autogen.sh 1>&2
    log "Building zfs-$ZFS..."
    ./configure --enable-linux-builtin=yes --enable-debug=yes --with-linux=$KERNEL_FULL_PATH --with-linux-obj=$KERNEL_FULL_PATH 1>&2

    log "copy-builtins"
    ./copy-builtin $KERNEL_FULL_PATH 1>&2
    log "Compiling zfs"
    make -j $CPU_CORES

I get:

make[3]: Entering directory 'zfsfolder/lib/libicp'
Makefile:1105

/bin/bash ../../libtool --silent --silent  --mode=compile gcc -DHAVE_CONFIG_H -include ../../zfs_config.h -I../../include -I../../include -I../../module/icp/include -I../../lib/libspl/include -I../../lib/libspl/include/os/linux   -D_GNU_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -DLIBEXECDIR=\"/usr/libexec\" -DRUNSTATEDIR=\"/usr/var/run\" -DSBINDIR=\"/usr/sbin\" -DSYSCONFDIR=\"/etc\" -DPKGDATADIR=\"/usr/share/zfs\" -UDEBUG -DNDEBUG  -DTEXT_DOMAIN=\"zfs-linux-user\"    -g -O2 -MT asm-x86_64/aes/aes_amd64.lo -MD -MP -MF $depbase.Tpo -c -o asm-x86_64/aes/aes_amd64.lo ../../module/icp/asm-x86_64/aes/aes_amd64.S &&\
mv -f $depbase.Tpo $depbase.Plo
  CPPAS    asm-x86_64/aes/aes_amd64.lo
In file included from ../../module/icp/include/sys/asm_linkage.h:32:0,
                 from ../../module/icp/asm-x86_64/aes/aes_amd64.S:204:
../../module/icp/include/sys/ia32/asm_linkage.h:32:27: fatal error: linux/linkage.h: No such file or directory
 #include <linux/linkage.h>

I kept building by adding manually the include corresponding to the include of the kernel build folder in the offending line.

stale[bot] commented 1 year ago

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.