Switching to more recent kernel

[ghostbuster91]

Hi,

This repository currently uses fixed version of linux kernel - "6.4.0-rc5". As "6.4.0" has been recently release I would like to switch to it.

Now, as you explained in another issue, we cannot just switch to the upstream kernel as we need to patch few things:

• dto
• nvme patch
• kernel config

dto

Based on the 6.4 tag it seems that some of them are no longer needed:

• mt7986a-bananapi-bpi-r3-nand.dts
• mt7986a-bananapi-bpi-r3-sd.dts

Am I right?

nvme patch

This is still required as the affected file hasn't been changed for quite some time. No action needed.

kernel config

This is the one I am the most puzzled about. This current config was generated for 6.3.0-rc2 so it does not even match the version of linux kernel that is in use. I guess that is not a big issue as this config is likely compatible between minor versions. Is it enough to leave it as it is, or should we regenerate new one for 6.4? If so how do one go about that and what options should be changed from the default one?

[nakato]

dto

The device tree overlays have been in the upstream kernel since 6.3, so the example provided here never needed to vendor them in, but as I was compiling on a RPi4 with an SD card and /tmp being on that SD card I didn't want to deal with the resource utilisation and time required to unpack and then unlink the entire source tree, twice(?). I think it added 10-15 minutes onto the ~60m it took to realise the updated system derivation after changing the kernel in any way. Plus the system would be rather unresponsive during this period as everything was stuck in iowait.

It's less of an issue if you're not testing new patches without a way to leverage incremental building, not building very often, or have a better build host.

Now that I think about it, the above is probably all moot as there is a way to apply binary device tree overlays in nixos that I didn't know about then, so something like this should work, but I haven't tried it yet.

  hardware.deviceTree.overlays = [
    {
      name = "bpir3-sd-enable";
      dtboFile = "${config.boot.kernelPackages.kernel}/dtbs/mediatek/mt7986a-bananapi-bpi-r3-sd.dtbo";
    }
    ...
  ];

kernel config

The kernel config being older isn't an issue as long as the defaults on any new options are okay and you don't need to enable anything that changed manually. For example, the following changed, but are enabled by default, so it was fine.

 CONFIG_NET_DSA_MT7530=y
+CONFIG_NET_DSA_MT7530_MDIO=y
+CONFIG_NET_DSA_MT7530_MMIO=y

I updated my kernel config with 6.4-rc5 a few weeks ago, so I've copied that one here.

nvme patch

It may not be required with every PCIe device and a BPiR3 board revision might fix the issue, but my bargain bin NVMe drives need it with the board revision I have (v1.1).

[nakato]

I should probably reduce how minimised the kernel config is now that I'm not re-compiling it constantly to improve compatibility. For example, to use a SFP you need to know what PHY it has and enable it in the kernel config to support it. I should let that section take the kernel defaults for example.

On the SFP note, I would avoid the 2.5GB RJ45 SFP modules that identify themselves as "OEM SFP-2.5G-T", such as the ones sold as "Compatible with BPI-R3 Routing Board", they're really odd and I've written off the ones I got as unusable.

[ghostbuster91]

As always, thanks for the detailed answer :)

The device tree overlays have been in the upstream kernel since 6.3

Hmm when I compare definitions in this repository and mainline kernel it seems for me that they are not the same. For example the wifi definition is missing in the mainline: https://github.com/torvalds/linux/tree/v6.4/arch/arm64/boot/dts/mediatek

I want to genuinely understand that, as I am going to run bpir3 as my main router and I want to minimize the risk of depending solely on you when it comes to kernel updates.

I updated my kernel config with 6.4-rc5 a few weeks ago, so I've copied that one here.

Awesome, thanks!

to use a SFP you need to know what PHY it has and enable it in the kernel config to support it. I should let that section take the kernel defaults for example.

That would be very helpful.

On the SFP note, I would avoid the 2.5GB RJ45 SFP modules that identify themselves as "OEM SFP-2.5G-T", such as the ones sold as "Compatible with BPI-R3 Routing Board", they're really odd and I've written off the ones I got as unusable.

Oh, that is great to know. I was going buy one to perf test the cable connection. I later found the topic where you discussed that with Frank on bananapi forum. Seems like there are few that have been tested and work well. I will try to get one.

[nakato]

The wifi eeprom-data will never be in the upstream kernel. It was originally submitted with the device-tree for the board and was removed. Removal request

With NixOS device-tree overlays are applied directly to the device-tree binary the kernel produces after the kernel derivation has been fully realised, creating a new derivation with the overlays applied; that is, they don't require recompiling the kernel to apply. You need overlays any time you want to add hardware, such as adding an RTC on the I2C bus, or change how hardware is configured, such as enabling/disabling UART, and are applied at various points. The ones applied by NixOS are applied pre-boot to the device-tree file firmware will load for Linux to use. Other changes to the device-tree will be made by boot firmware, u-boot in this case, during the boot process, such as chosen/uboot,version, chosen/linux,initrd-start, chosen/linux,initrd-end, etc. The u-boot in this repo is also configured to set the two ethernet MAC addresses to something static, as well as modifying the aforementioned eeprom-data to inject MAC addresses there as well as these devices do not come with any MAC addresses assigned.

graph LR;
    in-tree-device.dts-->in-tree-device.dtb;
    in-tree-overlay.dtso-->in-tree-overlay.dtbo;
    out-of-tree-overlay.dtso-->out-of-tree-overlay.dtbo;
    in-tree-device.dtb-->patched.dtb
    in-tree-overlay.dtbo-->patched.dtb
    patched.dtb-->patched.dtb'
    out-of-tree-overlay.dtbo-->patched.dtb'
    patched.dtb'-->in-memory
    u-boot-->in-memory
    in-memory-->in-memory'
    u-boot-->in-memory'
    in-memory'-->in-memory''
    u-boot-->in-memory''
    in-memory''-->in-memory'''
    u-boot-->in-memory'''
    u-boot-->linux
    in-memory'''-->linux

On non-nixos systems, the file-based overlays would be copied to the boot partition with the dtb, and u-boot would apply the overlays based on a board-global setting. A global setting that changes how all prior and future generations behave is not be deterministic. To keep boot generations deterministic, the overlays are thus pre-applied, and the updated device-tree copied to a derivation specific location in the boot partition.

[ghostbuster91]

Thank you, I understand that now.

So, in theory the only dto that will be eventually in the repository is the one with eeprom-data for wifi mt7986a-bananapi-bpi-r3-wirless.dts and the other ones are only kept here for your convenience?

Last but not least, how do you generated this kernel config file for new kernels?

[nakato]

There's also mt7986a-bananapi-bpi-r3-pcie-button.dts, which will remain, unless someone disables the reset button upstream. I'm not in the right head-space to do that myself right now, so that's not going to be me.

The reset button mistakenly uses one of the same pins that the PCIe port does, which makes pinctrl unhappy and refuses the pin to the pcie driver. You'll see https://elixir.bootlin.com/linux/v6.4/source/drivers/pinctrl/core.c#L211 in the kernel log.

Then there's mt7986a-efuse-device-tree-node.dts if you want /sys/devices/platform/soc/11d00000.efuse/nvmem0/nvmem. I was going to use that, but ended up not and placing my hacks into u-boot.

The kernel config can be updated by entering a development environment and running make oldconfig. If you want menuconfig you need to add ncurses to the env.

kernel = kernelPackages.kernel.overrideAttrs (oldAttrs: {
  nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [ pkg-config ncurses ];
});

[ghostbuster91]

Thanks! Sounds like I have everything that I needed.