Broken kernels cannot be "unbroken" with overlays.

[YellowOnion]

Describe the bug

sudo nixos-rebuild switch
building Nix...
building the system configuration...
error: 
Failed assertions:
- CONFIG_DEVTMPFS is not enabled!
- CONFIG_CGROUPS is not enabled!
- CONFIG_INOTIFY_USER is not enabled!
- CONFIG_SIGNALFD is not enabled!
- CONFIG_TIMERFD is not enabled!
- CONFIG_EPOLL is not enabled!
- CONFIG_NET is not enabled!
- CONFIG_SYSFS is not enabled!
- CONFIG_PROC_FS is not enabled!
- CONFIG_FHANDLE is not enabled!
- CONFIG_CRYPTO_USER_API_HASH is not enabled!
- CONFIG_CRYPTO_HMAC is not enabled!
- CONFIG_CRYPTO_SHA256 is not enabled!
- CONFIG_DMIID is not enabled!
- CONFIG_AUTOFS4_FS is not enabled!
- CONFIG_TMPFS_POSIX_ACL is not enabled!
- CONFIG_TMPFS_XATTR is not enabled!
- CONFIG_SECCOMP is not enabled!
- CONFIG_TMPFS is not yes!
- CONFIG_BLK_DEV_INITRD is not yes!
- CONFIG_EFI_STUB is not yes!
- CONFIG_BINFMT_ELF is not yes!
- CONFIG_UNIX is not enabled!
- CONFIG_INOTIFY_USER is not yes!
- CONFIG_NET is not yes!
- CONFIG_SWAP is not yes!
- Option driSupport32Bit requires a kernel that supports 32bit emulation
(use '--show-trace' to show detailed location information)

Note for some reason nixpkgs hasn't been updated:

nix-repl> pkgs = (import <nixos> {})
nix-repl> pkgs.linux_5_15
error: attribute 'linux_5_15' missing, at (string):1:1

Expected behavior

succeeds in building like 21.05

Notify maintainers

I don't know who to contact.

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

nix-shell -p nix-info --run "nix-info -m" 
these paths will be fetched (0.00 MiB download, 0.00 MiB unpacked):
  /nix/store/dsfb56vgbbgasffh2whnc6xyz068iy5h-nix-info
copying path '/nix/store/dsfb56vgbbgasffh2whnc6xyz068iy5h-nix-info' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 5.13.19, NixOS, 21.05.3964.6c0c3014634 (Okapi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.16`
 - channels(daniel): `""`
 - channels(root): `"nixos-21.11pre328987.c935f5e0add, nixos-unstable, nixpkgs-master"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module: hardware.opengl

[Artturin]

Send your config

[YellowOnion]

Send your config

Sorry I should have sent this earlier, I was trying to bisect it to see if I could pinpoint the problem.

I think it's because the bcachefs kernel is marked as broken in unstable.

https://github.com/YellowOnion/nixos-config

[YellowOnion]

Update:

Disabling the kernel overlays, driSupport, and enabling broken packages still end up with the CONFIG errors, but the driSupport error goes away.

Update 2: So I'm gonna go on a hunch and say that assertions are thrown before overlays are applied, so you can't even replace a broken kernel?

I wonder if #144930 will fix this.

Update 3: yes broken kernels throw assertion before overlays are applied. above pull request fixed this.

[ZoomRmc]

So what's the way out of this broken state, in case you can't boot earlier kernel?

[YellowOnion]

It looks like an override to linux_testing_bcachefs in overlays aren't working at all, I've been banging my head against a wall wondering why my patches are being ignored.

And because the kernel version is hard coded (no way to "force" my package with boot.kernelPackage) I'm stuck with an unusable system until the latest bcachefs is merged upstream.

@Artturin

Can someone try overriding the version string of their kernel in an overlay and see if nixos-rebuild actually bothers to change it or starts a rebuild with the old version?

[Artturin]

  boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.linux_5_10.override {
    argsOverride = rec {
      src = pkgs.fetchurl {
        url = "mirror://kernel/linux/kernel/v5.x/linux-${version}.tar.xz";
        sha256 = "sha256-R/zqmWwMAeWKxfhS/Cltd6NJbFUPMDQVL+536Drjj9o=";
      };
      version = "5.12.15";
      modDirVersion = "5.12.15";
    };
  });

have you tried something like this ( used my doc-repos script to find it https://github.com/Artturin/nixos-related-things doc-repos "linux.+override" )

https://nixos.wiki/wiki/Nixpkgs/Reviewing_changes#Pre-flakes this may work

[YellowOnion]

@Artturin The problem is that I need to override linux_testing_bcachefs because the bcachefs module has boot.kernelPackages hard coded to that. If I try the above it throws an error message.

Specifying mkForce 0 customKernelPkg seems to fix the error, but that seems kinda hacky?

Upon further inspection, linux_5_15 isn't exported by nixpkgs, linux_latest.version shows v5.15.1.

[hyperfekt]

@YellowOnion The Linux kernels were recently moved into the linuxKernel subattribute, with the top-level ones being merely aliases. The bcachefs module uses linuxPackages_testing_bcachefs, which is now an alias for linuxKernel.packages.linux_testing_bcachefs which uses linuxKernel.kernels.linux_testing_bcachefs. You can override them by overriding that linuxKernel attribute, like so:

(final: super: {
  linuxKernel = super.linuxKernel // {
    kernels = super.linuxKernel.kernels // {
      linux_testing_bcachefs = [...];
    };
  };
} )

[YellowOnion]

@hyperfekt sadly I get:

error: attribute 'override' missing, at /nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/system/boot/kernel.nix:41:18