generate-config.pl does not work well with boot.kernelPatches.extraConfig

[nh2]

Issue description

I have the fealing that the generate-config.pl script is pretty bugged, it keeps failing with repeated question and spews out tons of Error in reading or end of file. afterwards when trying to use boot.kernelPatches.extraConfig:

Output:

QUESTION:     USB Test and Measurement Class support, NAME: USB_TMC, ALTS: N/m/?, ANSWER: m
GOT: 
GOT:     *
GOT:     * NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
GOT:     *
GOT:     *
GOT:     * also be needed; see USB_STORAGE Help for more info
GOT:     *
QUESTION:     USB Mass Storage support, NAME: USB_STORAGE, ALTS: M/n/?, ANSWER: y
GOT: 
GOT: 
GOT: CONFIG_USB_STORAGE:
GOT: 
GOT: Say Y here if you want to connect USB mass storage devices to your
GOT: computer's USB port. This is the driver you need for USB
GOT: floppy drives, USB hard disks, USB tape drives, USB CD-ROMs,
GOT: USB flash devices, and memory sticks, along with
GOT: similar devices. This driver may also be used for some cameras
GOT: and card readers.
GOT: 
GOT: This option depends on 'SCSI' support being enabled, but you
GOT: probably also need 'SCSI device support: SCSI disk support'
GOT: (BLK_DEV_SD) for most USB storage devices.
GOT: 
GOT: To compile this driver as a module, choose M here: the
GOT: module will be called usb-storage.
GOT: 
GOT: Symbol: USB_STORAGE [=m]
GOT: Type  : tristate
GOT: Prompt: USB Mass Storage support
GOT:   Location:
GOT:     -> Device Drivers
GOT:       -> USB support (USB_SUPPORT [=y])
GOT:         -> Support for Host-side USB (USB [=m])
GOT:   Defined at drivers/usb/storage/Kconfig:8
GOT:   Depends on: USB_SUPPORT [=y] && USB [=m] && SCSI [=y]
GOT: 
GOT: 
GOT: 
QUESTION:     USB Mass Storage support, NAME: USB_STORAGE, ALTS: M/n/?, ANSWER: y
repeated question:     USB Mass Storage support at /nix/store/l6m0lgcrls587pz0i644jhfjk6lyj55s-generate-config.pl line 81.

Error in reading or end of file.

Error in reading or end of file.

Error in reading or end of file.

Error in reading or end of file.

and lots more of that, eventually failing with builder for '/nix/store/jjmxf3ivfp7xbz0y9bcqd9swvfrrc6h9-linux-config-4.4.178.drv' failed with exit code 255.

Indeed, it asked a question twice, should the script handle that if stuff is being patched?

Steps to reproduce

  boot.kernelPatches = [ {
    name = "chromiumos-inspired";
    patch = null;
    # https://cateee.net/lkddb/web-lkddb/BLK_DEV_SD.html
    #   Do not compile this driver as a module if your root file system
    #   (the one containing the directory /) is located on a SCSI disk.
    #   In this case, do not compile the driver for your SCSI host
    #   adapter (below) as a module either.
    extraConfig = ''
      SCSI y
      SCSI_MOD y
      BLK_DEV_SD y
      SCSI_SPI_ATTRS y
      ATA y
      SATA_AHCI y
      ATA_GENERIC y

      BLK_DEV_DM y
      DM_BUFIO y
      DM_BIO_PRISON y
      DM_PERSISTENT_DATA y
      DM_CRYPT y
      DM_THIN_PROVISIONING y
      DM_VERITY y

      USB_NET_DRIVERS y

      INPUT_EVDEV y

      KEYBOARD_CROS_EC y

      SERIO y
      SERIO_I8042 y
      SERIO_LIBPS2 y

      HW_RANDOM y
      HW_RANDOM_TPM y
      HW_RANDOM_INTEL y
      NVRAM y
      TCG_TPM y
      TCG_TIS y
      DEVPORT y

      ITCO_WDT y
      ITCO_VENDOR_SUPPORT y

      SSB_SDIOHOST y

      AGP y
      AGP_INTEL y
      INTEL_GTT y
      DRM y
      DRM_KMS_HELPER y

      FB_SYS_FILLRECT y
      FB_SYS_COPYAREA y
      FB_SYS_IMAGEBLIT y
      FB_SYS_FOPS y
      BACKLIGHT_GENERIC y

      #USB_STORAGE y
      USB_STORAGE_REALTEK y
      USB_UAS y
      USB_SERIAL y

      MMC y
      MMC_BLOCK y
      MMC_BLOCK_MINORS 16

      MMC_SDHCI y
      MMC_SDHCI_PCI y
      MMC_SDHCI_ACPI y
      LEDS_CLASS y

      DMA_OF y

      ASHMEM y
      ANDROID_TIMED_OUTPUT y
      ANDROID_TIMED_GPIO y
      SYNC y
      SW_SYNC y
      SW_SYNC_USER y
      ION y
      ION_DUMMY y
      ACPI_WMI y
      MXM_WMI y

      CHROMEOS_LAPTOP y
      CHROMEOS_PSTORE y
      CROS_EC_CHARDEV y
      CROS_EC_LPC y

      GOOGLE_FIRMWARE y

      EFI_VARS y
      EFI_VARS_PSTORE y
      EXT4_FS y
      EXT4_USE_FOR_EXT2 y
      EXT4_ENCRYPTION y
      JBD2 y
      FS_MBCACHE y
      AUTOFS4_FS y

      PSTORE_RAM y

      ENCRYPTED_KEYS y

      LSM_MMAP_MIN_ADDR 32768
      SECURITY_SELINUX_BOOTPARAM_VALUE 1
      DEFAULT_SECURITY "selinux"
    '';

Technical details

On commit 05a53ecd827c5910e34171e7681c56913b29add1

[nh2]

Also, what does this mean?

error: option not set correctly: USB_NET_DRIVERS (wanted 'y', got 'm')

Yes, if I write USB_NET_DRIVERS y into extraConfig, then I really do want y!

Am I misunderstanding this?

[teto]

USB_NET_DRIVERS may depend on other settings set to "m" which prevents USB_NET_DRIVERS y, that's why the script can't answer "y" and fails warning you. Thus you have to search for USB_NET_DRIVERS dependencies and set them to "y" as well.

[nh2]

USB_NET_DRIVERS may depend on other settings set to "m" which prevents USB_NET_DRIVERS y, that's why the script can't answer "y" and fails warning you. Thus you have to search for USB_NET_DRIVERS dependencies and set them to "y" as well.

Ah, I see. That makes sense.

[evanjs]

Not sure if it's related, but I'm experiencing similar issues, albeit when overriding a musnix kernel (4.19.31) with structuredExtraOptions.

I'm trying to enable TRANSPARENT_HUGEPAGES, and the option never even appears in the list of questions.

The other options on the Memory Management page appear like they should. I checked the kernel source that is being used and the option does seem to exist 😕

[teto]

you can check in a nix-shell how the configuration is generated, warnings appear at the end when nix was nox able to select a requested config item eval "$configurePhase"

[evanjs]

you can check in a nix-shell how the configuration is generated, warnings appear at the end when nix was not able to select a requested config item eval "$configurePhase"

I suppose I forgot to mention I'm using NixOps. Is there an easy way to initiate a nix-shell for NixOps deployments? Or would it be easiest to debug in a different manner?

[teto]

you should run the nix-shell with the kernel version that is built by nixops for your machine. nixops builds it locally before sending the closure on the target so nixops doesn't make a difference.

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[nh2]

still important to me

[snajpa]

Hitting this too with linux 5.8-rc2:

GOT: Defined at drivers/net/wireless/realtek/rtw88/Kconfig:28
GOT:   Prompt: Realtek 8822BE PCI wireless network adapter
GOT:   Depends on: NETDEVICES [=y] && WLAN [=y] && WLAN_VENDOR_REALTEK [=y] && RTW88 [=m] && PCI [=y]
GOT:   Location:
GOT:     -> Device Drivers
GOT:       -> Network device support (NETDEVICES [=y])
GOT:         -> Wireless LAN (WLAN [=y])
GOT:           -> Realtek devices (WLAN_VENDOR_REALTEK [=y])
GOT:             -> Realtek 802.11ac wireless chips support (RTW88 [=m])
GOT: Selects: RTW88_CORE [=n] && RTW88_PCI [=n] && RTW88_8822B [=n]
GOT:
GOT:
GOT:
QUESTION:         Realtek 8822BE PCI wireless network adapter, NAME: RTW88_8822BE, ALTS: N/m/?, ANSWER: y
repeated question:         Realtek 8822BE PCI wireless network adapter at /nix/store/ri66gafl5xrrqkf5jssjpj10xd1jad3n-generate-config.pl line 81.

Error in reading or end of file.

Error in reading or end of file.

Error in reading or end of file.

[stale[bot]]

I marked this as stale due to inactivity. → More info

[nh2]

still

[JenniferPadilla]

@snajpa I'm hitting the exact same repeated question while bisecting for a bug introduced between 5.7 and 5.8 that makes my VMs hang for minutes at a time. Is there anything I can define so generate-config.pl answers with, say, m, as, according to its comments, it's attempting to do anyway?

[nh2]

@JenniferPadilla I don't have the answer for you, but when I had to bisect a kernel (that's also how I found this bug), I worked around it by bisecting + building "outside of nix", that is, I prepared the build environment nix-shell once, and ran make -j in git bisect, to quickly build successive kernels, instead of doing a full nix-build every time.

[teto]

Like nh2. also we should use a structured extraConfig like is done in the rest of the kernel configuration and deprecate the legacy "extraConfig" setting. Otherwise we lose an opporunity to detect conflicts in the nix configuration.

[JenniferPadilla]

@nh2 I found that if you add kernel configuration items to an extraConfig key of any boot.kernelPatches member, just as if you'd add any old kernel config for your own purposes, then that's already sufficient to override the wrong guesses generate-config.pl makes. So just add RTW88_8822BE and RTW88_8822CE as appropriate.

[milahu]

spews out tons of Error in reading or end of file

same with linuxManualConfig

https://github.com/NixOS/nixpkgs/blob/d917b8cdb3f692e604594d7732acd88eb684c3d1/pkgs/os-specific/linux/kernel/manual-config.nix#L161-L163

but i dont see how the interactive prompt is answered 0__o when i run make oldconfig in a terminal, it blocks waiting for my input

it keeps failing with repeated question

not here. in my case the build continues

[stale[bot]]

I marked this as stale due to inactivity. → More info

[VergeDX]

still, I'm packaging linux with custom source.

[VergeDX]

Update: Disable autoModules in buildLinux works fine for me.

But it seems this will cause different derivation calculation between "build with nix only" and "build on NixOS"? Some options will be true on NixOS, but false on "nix only" build.

[shelvacu]

I think I understand what's going on here. I had this issue and fixed it by changing some =lib.kernel.yes options to =lib.kernel.module. The problem was that a prerequisite option was set to module, and so answering y is invalid. You can see what option is having causing problems (one at a time) by scrolling up to the first message before all the Error in reading or end of file.s

You can see this in the original problem:

USB Mass Storage support, NAME: USB_STORAGE, ALTS: M/n/?, ANSWER: y

M/n/? are the possible options, module, no, or ? (help). But then generate-config gave it ANSWER: y and then everything breaks

[shelvacu]

USB_NET_DRIVERS may depend on other settings set to "m" which prevents USB_NET_DRIVERS y, that's why the script can't answer "y" and fails warning you. Thus you have to search for USB_NET_DRIVERS dependencies and set them to "y" as well.

The answer was here the whole time. What is this issue about exactly? A better error message?

[asonix]

Just ran into a similar issue where I had to use the kernelPatches extraConfig workaround.

the configuration script kept reading DRM_PANEL_BOE_TH101MB31UIG002_28A before its dependency BACKLIGHT_CLASS_DEVICE and would fail.