mount: /sys/fs/bpf: permission denied.

nueavv commented 8 months ago

What happened: failed to install cilium cni. the mount-bpf-fs container log in cilium pod is "mount: /sys/fs/bpf: permission denied."

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
- role: worker
networking:
  disableDefaultCNI: true
  kubeProxyMode: none

First, create a cluster with this configuration on mac. and then install cilium cni with cilium cli. the cilium version is 1.15.1

Anything else we need to know?:

Environment:

kind version: (use kind version): kind v0.22.0 go1.21.7 darwin/arm64

Runtime info: (use docker info, podman info or nerdctl info):

host:
arch: arm64
buildahVersion: 1.33.5
cgroupControllers:
- cpu
- io
- memory
- pids
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: conmon-2.1.10-1.fc39.aarch64
path: /usr/bin/conmon
version: 'conmon version 2.1.10, commit: '
cpuUtilization:
idlePercent: 94.35
systemPercent: 2.78
userPercent: 2.87
cpus: 4
databaseBackend: sqlite
distribution:
distribution: fedora
variant: coreos
version: "39"
eventLogger: journald
freeLocks: 2040
hostname: localhost.localdomain
idMappings:
gidmap:
- container_id: 0
  host_id: 1000
  size: 1
- container_id: 1
  host_id: 100000
  size: 1000000
uidmap:
- container_id: 0
  host_id: 2048452658
  size: 1
- container_id: 1
  host_id: 100000
  size: 1000000
kernel: 6.7.5-200.fc39.aarch64
linkmode: dynamic
logDriver: journald
memFree: 72327168
memTotal: 2044411904
networkBackend: netavark
networkBackendInfo:
backend: netavark
dns:
  package: aardvark-dns-1.10.0-1.fc39.aarch64
  path: /usr/libexec/podman/aardvark-dns
  version: aardvark-dns 1.10.0
package: netavark-1.10.3-1.fc39.aarch64
path: /usr/libexec/podman/netavark
version: netavark 1.10.3
ociRuntime:
name: crun
package: crun-1.14.3-1.fc39.aarch64
path: /usr/bin/crun
version: |-
  crun version 1.14.3
  commit: 1961d211ba98f532ea52d2e80f4c20359f241a98
  rundir: /run/user/2048452658/crun
  spec: 1.0.0
  +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
os: linux
pasta:
executable: /usr/bin/pasta
package: passt-0^20240220.g1e6f92b-1.fc39.aarch64
version: |
  pasta 0^20240220.g1e6f92b-1.fc39.aarch64-pasta
  Copyright Red Hat
  GNU General Public License, version 2 or later
    <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.
remoteSocket:
exists: true
path: /run/user/2048452658/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: true
serviceIsRemote: true
slirp4netns:
executable: /usr/bin/slirp4netns
package: slirp4netns-1.2.2-1.fc39.aarch64
version: |-
  slirp4netns version 1.2.2
  commit: 0ee2d87523e906518d34a6b423271e4826f71faf
  libslirp: 4.7.0
  SLIRP_CONFIG_VERSION_MAX: 4
  libseccomp: 2.5.3
swapFree: 0
swapTotal: 0
uptime: 0h 10m 20.00s
variant: v8
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
- ipvlan
volume:
- local
registries:
search:
- docker.io
store:
configFile: /var/home/core/.config/containers/storage.conf
containerStore:
number: 4
paused: 0
running: 4
stopped: 0
graphDriverName: overlay
graphOptions: {}
graphRoot: /var/home/core/.local/share/containers/storage
graphRootAllocated: 106769133568
graphRootUsed: 8191610880
graphStatus:
Backing Filesystem: xfs
Native Overlay Diff: "true"
Supports d_type: "true"
Supports shifting: "false"
Supports volatile: "true"
Using metacopy: "false"
imageCopyTmpDir: /var/tmp
imageStore:
number: 1
runRoot: /run/user/2048452658/containers
transientStore: false
volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
APIVersion: 4.9.3
Built: 1708357248
BuiltTime: Tue Feb 20 00:40:48 2024
GitCommit: ""
GoVersion: go1.21.7
Os: linux
OsArch: linux/arm64
Version: 4.9.3

OS (e.g. from /etc/os-release):

Kubernetes version: (use kubectl version):

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-04-14T13:14:41Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"29", GitVersion:"v1.29.2", GitCommit:"4b8e819355d791d96b7e9d9efe4cbafae2311c88", GitTreeState:"clean", BuildDate:"2024-02-14T22:25:42Z", GoVersion:"go1.21.7", Compiler:"gc", Platform:"linux/arm64"}
WARNING: version difference between client (1.27) and server (1.29) exceeds the supported minor version skew of +/-1

Any proxies or other special environment settings?:

BenTheElder commented 8 months ago

Er, cilium CNI isn't something we officially support, but it should work as I understand them to be using KIND for some of their CI.

Can you please test a minimal config first? Also in most cases you should not need multiple nodes unless you're testing some really specific multi-node related behavior.

BenTheElder commented 8 months ago

I think cilium is probably better equipped to support this based on https://docs.cilium.io/en/stable/installation/kind/, I do not work with cilium or bpf. A sufficiently privileged pod should be able to mount this.

I noticed you're using podman, which is semi-experimental still in kind and I'm not sure if the cilium project supports this.

nueavv commented 8 months ago

Okay, then I will try docker.

BenTheElder commented 7 months ago

Did it work with docker and/or following cilium's docs?

nueavv commented 7 months ago

yes, It works! thank you

BenTheElder commented 7 months ago

Good! I guess there is something off with this mount on the podman install, for now will close this 😅

kubernetes-sigs / kind

mount: /sys/fs/bpf: permission denied. #3545