GPL-incompatible module zfs.ko uses GPL-only symbol 'migrate_enable'

[szanni]

System information

Type	Version/Name
Distribution Name	ArchLinux
Distribution Version
Linux Kernel	5.9.1.18.arch1-2 RT (Real Time)
Architecture	x86_64
ZFS Version	0.8.5
SPL Version

Describe the problem you're observing

Trying to compile zfs 0.8.5 against the ArchLinux real time kernel linux-rt version 5.9.1.18.arch1-2 fails with:

  MODPOST /home/user/repo/aur/zfs-linux-rt/src/zfs-0.8.5/module/Module.symvers
FATAL: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'migrate_enable'

This is most likely due to PREEMPTION=y as I found numerous bug reports regarding other GPL-only symbols. This regards a different symbol though, hence a separate bug report.

Describe how to reproduce the problem

Install linux-rt, linux-rt-headers, and zfs-utils from the AUR. Then save the following contents into a file named PKGBUILD and run makepkg.

pkgbase="zfs-linux-rt"
pkgname=("zfs-linux-rt" "zfs-linux-rt-headers")
_zfsver="0.8.5"
_kernelver="5.9.1.18.arch1-2"
_extramodules="5.9.1-rt18-2-rt"

pkgver="${_zfsver}_$(echo ${_kernelver} | sed s/-/./g)"
pkgrel=1
makedepends=("linux-rt-headers=${_kernelver}")
arch=("x86_64")
url="https://zfsonlinux.org/"
source=("https://github.com/zfsonlinux/zfs/releases/download/zfs-${_zfsver}/zfs-${_zfsver}.tar.gz")
sha256sums=("dbb41d6b9c606a34ac93f4c19069fd6806ceeacb558f834f8a70755dadb7cd3d")
license=("CDDL")
depends=("kmod" "zfs-utils=${_zfsver}" "linux-rt=${_kernelver}")

build() {
    cd "${srcdir}/zfs-${_zfsver}"
    ./autogen.sh
    ./configure --prefix=/usr --sysconfdir=/etc --sbindir=/usr/bin --libdir=/usr/lib \
                --datadir=/usr/share --includedir=/usr/include --with-udevdir=/usr/lib/udev \
                --libexecdir=/usr/lib --with-config=kernel \
                --with-linux=/usr/lib/modules/${_extramodules}/build \
                --with-linux-obj=/usr/lib/modules/${_extramodules}/build
    make
}

package_zfs-linux-rt() {
    pkgdesc="Kernel modules for the Zettabyte File System."
    install=zfs.install
    provides=("zfs" "spl")
    groups=("archzfs-linux-rt")
    conflicts=("zfs-dkms" "zfs-dkms-git" "zfs-dkms-rc" "spl-dkms" "spl-dkms-git")
    cd "${srcdir}/zfs-${_zfsver}"
    make DESTDIR="${pkgdir}" INSTALL_MOD_PATH=/usr install
    # Remove src dir
    rm -r "${pkgdir}"/usr/src
}

package_zfs-linux-rt-headers() {
    pkgdesc="Kernel headers for the Zettabyte File System."
    provides=("zfs-headers" "spl-headers")
    conflicts=("zfs-headers" "zfs-dkms" "zfs-dkms-git" "zfs-dkms-rc" "spl-dkms" "spl-dkms-git" "spl-headers")
    cd "${srcdir}/zfs-${_zfsver}"
    make DESTDIR="${pkgdir}" install
    rm -r "${pkgdir}/lib"
    # Remove reference to ${srcdir}
    sed -i "s+${srcdir}++" ${pkgdir}/usr/src/zfs-*/${_extramodules}/Module.symvers
}

[PrivatePuffin]

"This is most likely due to PREEMPTION=y as I found numerous bug reports regarding other GPL-only symbols." I don't think it's even relevant to open a bug report, as "PREEMPTION=y", just isn't supported.

It isn't a bug if it isn't supposed to work imho.

[szanni]

I don't think it's even relevant to open a bug report, as "PREEMPTION=y", just isn't supported.

Are you sure? I've found numerous bugs regarding PREEMPTION=y and fixes being merged like #10973.

That being said my issue is most likely due to PREEMPT_RT and not PREEMPTION itself, my mistake. I do understand that this is currently still an out of tree patch set which should however be mainlined soon. And it would be nice to not leave any pro audio users and others using RT kernels by the way side.

Looking at the META file - it does indeed state max kernel version 5.6 but when I fake change the license to GPL it does compile. I can however not distribute that (AUR maintainer here).

Trying to compile the latest git version which lists 5.9 as supported fails during the configure stage with:

checking whether struct rw_semaphore member wait_lock is raw... configure: error: 
    *** None of the expected "rwsem_spinlock_is_raw" interfaces were detected.
    *** This may be because your kernel version is newer than what is
    *** supported, or you are using a patched custom kernel with
    *** incompatible modifications.
    ***
    *** ZFS Version: zfs-2.0.0-rc1_147_g2aaab887b
    *** Compatible Kernels: 3.10 - 5.9

I do not expect this bug report to be high priority, but I do believe it to be valuable to raise attention that there are linux-rt users out there that would prefer not having to switch their systems to an inferior file system.

[PrivatePuffin]

@szanni Shoot, i think I misunderstood you. I understood you meant that "PREEMPTION=y" was one of the known unsupported flags, but considering your last comment that indeed doesn't seem the case...

0.8.5 does indeed have a wrong META file (with 5.6 listed instead of 5.9)

[rugubara]

It seems that the changes causing this are being back-ported. I have the same configure issue with linux-5.6.19-rt-p12 and rt-p11 on Gentoo.

[orivej]

linux-rt has a different definition of struct rw_semaphore:

struct rw_semaphore {
    atomic_t        readers;
    struct rt_mutex     rtmutex;
#ifdef CONFIG_DEBUG_LOCK_ALLOC
    struct lockdep_map  dep_map;
#endif
};

However zfs does not depend on the type or presence of rw_semaphore.wait_lock since about 4f8e643afe9d8cc558566e1dda05ef597cb2526e, so its configure check can be removed.

[orivej]

Fix for struct rw_semaphore member wait_lock is raw configure check: #11309

[orivej]

The only GPL-only symbols that come from linux-rt are migrate_disable and migrate_enable. They are used only in the expansion of the macros kmap_atomic and kunmap_atomic (in zcommon/zfs_uio.c and os/linux/zfs/abd_os.c).

[orivej]

I am asking linux-rt maintainers to export these functions: https://lore.kernel.org/linux-rt-users/20201208212841.694b3022@orivej.orivej.org/T/

[szanni]

With the PREEMPT_RT patches now (5.15) being mainline I decided to try again.

ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'migrate_enable'
ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'migrate_disable'

The licensing issue keeps persisting with kernel 5.15.2.19.realtime1-2 and zfs 2.1.1 but it does build and run if I fake change the ZFS license to GPL in the META file.

I decided to plead to the lkml again: https://lore.kernel.org/lkml/3bbacbed-2c52-f37f-070c-adece794e3ef@szanni.org/T/

[AnterCreeper]

This bus is still happens on zfs 2.1.6 with debian 5.18.0-0.deb11.4-rt-amd64

[satmandu]

Is there a workaround for this aside from locally changing the license? This is still a problem with the 6.3-rt kernel series and openzfs/master...

[szanni]

Is there a workaround for this aside from locally changing the license? This is still a problem with the 6.3-rt kernel series and openzfs/master...

I have sadly not come across any upstream fix so far. My hopes were that Ubuntu Studio would eventually run into the same problem and hopefully fix things - as they pay developers that work on the kernel - but so far no luck.

[satmandu]

For what it is worth, I had system instability (lockups) changing the license locally and running main on the 6.3-rt kernel, but I'm not sure that was necessarily due to OpenZFS, as I didn't try 6.3-rt without OpenZFS...

[ryao]

Is there a workaround for this aside from locally changing the license? This is still a problem with the 6.3-rt kernel series and openzfs/master...

I have not tried it, but ZFS_MODULE_CPPFLAGS="-Dmigrate_enable(...)=preempt_enable(__VA_ARGS__) -Dmigrate_disable(...)=preempt_disable(__VA_ARGS__)" might work.

If the RT kernel does not honor the preemption counter anymore, then this workaround would cause problems. :/

[szanni]

For what it is worth, I had system instability (lockups) changing the license locally and running main on the 6.3-rt kernel, but I'm not sure that was necessarily due to OpenZFS, as I didn't try 6.3-rt without OpenZFS...

I have encountered something similar myself with certain rt kernel versions. Seemingly all input failing right after boot (not sure if this is the lockup you were talking about). The next point release usually fixed things, so I never bothered investigating. My hunch was it being related to input drivers and/or xorg and the rt kernel rather than any file system module though, but who knows.

[wargreen]

I also run into this issue. For pro-audio we need both RT and good storage. Can we hope than, a day, we can run out of this license issue ?

[Harliff]

I've just got this error on Debian 12 with 6.1.0-13-rt kernel.

logs:

MODPOST /home/ilya/soft/zfs/module/Module.symvers
ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'migrate_disable'
ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'migrate_enable'
make[4]: *** [/usr/src/linux-headers-6.1.0-13-common-rt/scripts/Makefile.modpost:126: /home/ilya/soft/zfs/module/Module.symvers] Error 1
make[3]: *** [/usr/src/linux-headers-6.1.0-13-common-rt/Makefile:1991: modpost] Error 2
make[3]: Leaving directory '/usr/src/linux-headers-6.1.0-13-rt-amd64'
make[2]: *** [Makefile:56: modules-Linux] Error 2
make[2]: Leaving directory '/home/ilya/soft/zfs/module'
make[1]: *** [Makefile:12116: all-recursive] Error 1
make[1]: Leaving directory '/home/ilya/soft/zfs'
make: *** [Makefile:4600: all] Error 2

This solution does not work for me:

ZFS_MODULE_CPPFLAGS="-Dmigrate_enable(...)=preempt_enable(__VA_ARGS__) -Dmigrate_disable(...)=preempt_disable(__VA_ARGS__)" might work.

[Harliff]

but it does build and run if I fake change the ZFS license to GPL in the META file.

Yes, this is working workaround:

sed  's/CDDL/GPL/' -i META

[satmandu]

Now that Kernel 6.12-rc1, which includes build options for PREEMPT_RT without any additional patching, maybe someone might be able to help brainstorm how to deal with the GPL symbol issues with OpenZFS and PREEMPT_RT enabled?

(As of last night, the exact same GPL errors pop up during an attempted dkms install with a PREEMPT_RT enabled 6.12-rc1 kernel when I try to build OpenZFS kernel modules.)

[szanni]

Now that Kernel 6.12-rc1, which includes build options for PREEMPT_RT without any additional patching, maybe someone might be able to help brainstorm how to deal with the GPL symbol issues with OpenZFS and PREEMPT_RT enabled?

It would indeed be great to have a more permanent solution.

Pleading to the LKML has so far been fruitless. Would this be different as PREEMPT_RT is now mainline-rc? I sadly have my doubts, as kernel space is not user space and there is seemingly no rule not to break kernel space.