search

kubernetes-sigs / node-feature-discovery

Node feature discovery for Kubernetes
Apache License 2.0
757 stars 237 forks source link

Docs: failed to discover kernel config and builtin modules in the development environment started using tilt #1855

Open ChaoyiHuang opened 2 weeks ago

ChaoyiHuang commented 2 weeks ago

What happened: In the developer guide , when running with tilt, nfd worker will not be able to discover kernel config and builtin modules

the error message is: nfd-worker │ E0826 07:07:12.558316 1 kernel.go:134] "failed to read kconfig" err="failed to read kernel config from [ /proc/config.gz /host-usr/src/linux-6.5.0-1025-gcp/.config /host-usr/src/linux/.config /host-usr/lib/modules/6.5.0-1025-gcp/config /host-usr/lib/ostree-boot/config-6.5.0-1025-gcp /host-usr/lib/kernel/config-6.5.0-1025-gcp /host-usr/src/linux-headers-6.5.0-1025-gcp/.config /lib/modules/6.5.0-1025-gcp/build/.config /host-boot/config-6.5.0-1025-gcp]"

nfd-worker │ E0826 07:07:12.559161 1 kernel.go:149] "failed to get builtin kernel modules" err="failed to read file /host-lib/modules/6.5.0-1025-gcp/modules.builtin: open /host-lib/modules/6.5.0-1025-gcp/modules.builtin: no such file or directory"

What you expected to happen: no matter what OS is, kernel config and builtin modules should be discovered correctly

How to reproduce it (as minimally and precisely as possible):

  1. start a virtual machine with OS unbuntu 22.04
  2. following the developer guide

Anything else we need to know?:

Environment:

marquiz commented 2 weeks ago

/assign @fmuyassarov

ChaoyiHuang commented 2 weeks ago

the issue is due to the k3d/kind cluster created by ctlptl will run inside a container(it will serve as the virtual host).

Host folders which will be scaned by the nfd feature discovery should be mounted into the container ( the virtual host). otherwise the nfd-worker container which run inside the virtual host will just see the default base image rootfs /boot, /lib folders, which are usually empty, leads to the discovey failure.

For k3d, the following scripts can pass the host folder to the container, of course, it's better to make these folders read-only.

ctlptl create cluster kind --registry=ctlptl-registry

cat <<EOF | ctlptl apply -f -
apiVersion: ctlptl.dev/v1alpha1
kind: Cluster
registry: ctlptl-registry
product: k3d
k3d:
  v1alpha5Simple:
    volumes:
      - volume: /boot:/boot
        nodeFilters:
          - server:0
          - agent:*
      - volume: /lib:/lib
        nodeFilters:
          - server:0
          - agent:*
EOF