Is it possibility to enable br_netfilter module in kernel?

[plkumar]

I'm trying to get k3s working on gokrazy, it needs br_netfilter modules, is it possible to enable this?

I'm newbie to kernel configuration and compilation, so not sure if this is feasible or not.

[stapelberg]

Any module should be possible to enable.

See https://github.com/gokrazy/kernel#updating-the-kernel for instructions, and modify https://github.com/gokrazy/kernel/blob/bb2c877c18f813da87b28fbbf8f0599f689c4017/cmd/gokr-build-kernel/build.go#L19 to change the kernel configuration.

If the module requires other changes to the kernel config, you can unpack the latest Linux kernel’s source in a temporary directory, and diff the config before and after make menuconfig.

Let me know if you can’t get it to work

[plkumar]

i tried both by cloning this repo and forking it, and updated the configAddendum, and checked the .config file after kernel compilation and also in modules.builtin file, i could see br_netfilter module, but when i update to Pi, i don't see this module available on the Pi when i check with breakglass, I'm sure I'm doing something wrong but not sure where.

here is the command i used on my fork

gokr-packer -serial_console=disabled -kernel_package=github.com/plkumar/gokrazy-kernel \
  -update=yes \
  github.com/gokrazy/hello \
  github.com/gokrazy/breakglass \
  github.com/gokrazy/serial-busybox \
  github.com/gokrazy/iptables \
  github.com/gokrazy/nsenter \
  github.com/gokrazy/podman \
periph.io/x/cmd/periph-info

When i complie kernel locally using "go run ./cmd/gokr-build-kernel/build.go" in side the cloned folder, it compiles, but compiled output seems to be going to /tmp/buildresult

Here is the output

*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
#
# configuration written to .config
#
#
# configuration written to .config
#
  SYNC    include/config/auto.conf.cmd
*
* Restart config...
*
*
* ARMv8.3 architectural features
*
Enable support for pointer authentication (ARM64_PTR_AUTH) [Y/n/?] y
  Use pointer authentication for kernel (ARM64_PTR_AUTH_KERNEL) [Y/n/?] (NEW) 
*
* ARMv8.4 architectural features
*
Enable support for the Activity Monitors Unit CPU extension (ARM64_AMU_EXTN) [Y/n/?] y
Enable support for tlbi range feature (ARM64_TLB_RANGE) [Y/n/?] (NEW) 
*
* ARMv8.5 architectural features
*
Branch Target Identification support (ARM64_BTI) [Y/n/?] y
  Use Branch Target Identification for kernel (ARM64_BTI_KERNEL) [Y/n/?] (NEW) 
Enable support for E0PD (ARM64_E0PD) [Y/n/?] y
Enable support for random number generation (ARCH_RANDOM) [Y/n/?] y
Memory Tagging Extension support (ARM64_MTE) [Y/n/?] (NEW) 
  CALL    scripts/atomic/check-atomics.sh
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/crypto/cmac.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/crypto/ecc.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/crypto/ecdh_generic.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/drivers/bluetooth/btbcm.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/drivers/bluetooth/btqca.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/drivers/bluetooth/btqcomsmd.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/drivers/bluetooth/hci_uart.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/drivers/net/wireless/broadcom/brcm80211/brcmutil/brcmutil.ko
  INSTALL /tmp/buildresult/lib/modules/5.18.0/kernel/net/bluetooth/bluetooth.ko
  DEPMOD  /tmp/buildresult/lib/modules/5.18.0

Here is the grep result from modules.builtin

~/kernel/linux-5.18 cat modules.builtin | grep bridge
kernel/drivers/gpu/drm/bridge/display-connector.ko
kernel/drivers/gpu/drm/bridge/adv7511/adv7511.ko
kernel/drivers/gpu/drm/bridge/synopsys/dw-hdmi.ko
kernel/drivers/gpu/drm/bridge/synopsys/dw-hdmi-i2s-audio.ko
kernel/net/bridge/bridge.ko
kernel/net/bridge/br_netfilter.ko

[stapelberg]

It sounds like you’re building with br_netfilter=y (builtin) instead of br_netfilter=m (module)? If you’re not setting =y explicitly, possibly some dependencies (bridge?) are built as builtin instead of module?

[stapelberg]

Also, how are you checking if the module is available on the Pi specifically? What does /proc/config.gz say? And does gokr-packer print the correct directory for your kernel? (If not, you probably need a replace directive in your go.mod file.)

[plkumar]

It sounds like you’re building with br_netfilter=y (builtin) instead of br_netfilter=m (module)? If you’re not setting =y explicitly, possibly some dependencies (bridge?) are built as builtin instead of module?

I was setting the br_netfilter=y, after your reply I tried with br_netfilter=m as well.

Also, how are you checking if the module is available on the Pi specifically? What does /proc/config.gz say? And does gokr-packer print the correct directory for your kernel? (If not, you probably need a replace directive in your go.mod file.)

I used the replace directive, it is pointing to the right path (kernel repo on my local machine).

/proc/config.gz' is saying# CONFIG_BRIDGE_NETFILTER is not set`

/perm/configgz # cat config | grep BRIDGE
# CONFIG_BRIDGE_NETFILTER is not set
# CONFIG_NF_TABLES_BRIDGE is not set
# CONFIG_NF_CONNTRACK_BRIDGE is not set
# CONFIG_BRIDGE_NF_EBTABLES is not set
CONFIG_BRIDGE=y
CONFIG_BRIDGE_IGMP_SNOOPING=y
# CONFIG_BRIDGE_MRP is not set
# CONFIG_BRIDGE_CFM is not set
CONFIG_PCI_BRIDGE_EMUL=y
CONFIG_DRM_BRIDGE=y
CONFIG_DRM_PANEL_BRIDGE=y
# CONFIG_DRM_SIMPLE_BRIDGE is not set
# CONFIG_FPGA_BRIDGE is not set

[stapelberg]

I was setting the br_netfilter=y, after your reply I tried with br_netfilter=m as well.

You’re saying BR_NETFILTER here, but the option is called CONFIG_BRIDGE_NETFILTER.

When I enable CONFIG_BRIDGE_NETFILTER in ARCH=arm64 make menuconfig, I get the following diff in the .config tool:

CONFIG_BRIDGE_NETFILTER=m
CONFIG_NETFILTER_FAMILY_BRIDGE=y

Can you try these two lines as configAddendum?

[plkumar]

@stapelberg my mistake, I meant "CONFIG_BRIDGE_NETFILTER" only, earlier i had only "CONFIG_BRIDGE_NETFILTER=m", I tried again with "CONFIG_BRIDGE_NETFILTER=m" and "CONFIG_NETFILTER_FAMILY_BRIDGE=y" in configAddendum, but it did not work. it seems to pick up the right kernel when updating using gokr-packer. but I don't see the module on Pi, even k3s binary is complaining that it did not find the module.

[stapelberg]

When I change the code like so:

diff --git i/cmd/gokr-build-kernel/build.go w/cmd/gokr-build-kernel/build.go
index d11ce2f..c0ddc29 100644
--- i/cmd/gokr-build-kernel/build.go
+++ w/cmd/gokr-build-kernel/build.go
@@ -1040,6 +1040,8 @@ CONFIG_SND_SOC_TEGRA_ALC5632=y
 CONFIG_SND_SOC_TEGRA_MAX98090=y
 CONFIG_SND_SOC_TEGRA_RT5677=y

+CONFIG_BRIDGE_NETFILTER=m
+CONFIG_NETFILTER_FAMILY_BRIDGE=y
 `

 func downloadKernel() error {

And then run:

    go install ./cmd/...
    gokr-rebuild-kernel

I do see INSTALL /tmp/buildresult/lib/modules/5.18.3/kernel/net/bridge/br_netfilter.ko in the output, and that file is in my kernel directory afterwards. Is that not what you’re seeing?

[plkumar]

@stapelberg thank you, this time it worked, the module is loaded and modprobe br_netfilter worked.

k3s is still not working though, I had to build kernel with vxlan as well, now I'm having issues with read-only root filesystem, tried mounting some of the path to /perm/<dir>, still have other issues, need to troubleshoot.

Appreciate your support.

[stapelberg]

(Closing this old issue as the br_netfilter problem seems to be resolved.)