cnoe-io / idpbuilder

Spin up a complete internal developer platform with only Docker required as a dependency.
https://cloud-native.slack.com/archives/C05TN9WFN5S
Apache License 2.0
194 stars 59 forks source link

podman rootless support #356

Closed nabuskey closed 1 week ago

nabuskey commented 3 months ago

Need to document. Tested on RHEL 9. It works in rootful but doesn't work out of the box in rootless.

The iptables module is disabled by default so we need to enable it. But seems to work afterwards.

lsmod | grep ip_table
sudo modprobe ip_tables
echo 'ip_tables' | sudo tee -a /etc/modules-load.d/ip_tables.conf
lsmod | grep ip_table

error:

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "8d10df7cef38c6c7fbb21426f617b7b4e4df470c4207f07668fa90f7f008e6b9": plugin type="portmap" failed (add): unable to create chain CNI-HOSTPORT-SETMARK: failed to list iptables chains: running [/usr/sbin/iptables -t nat -S --wait]: exit status 3: modprobe: ERROR: could not insert 'ip_tables': Operation not permitted
iptables v1.8.9 (legacy): can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
cmoulliard commented 1 week ago

I'm doing a test using podman rootless on macos and got this error when we create a new cluster

running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

I followed the steps described here and added yes to the following file without success

cat /etc/systemd/system/user@.service.d/delegate.conf
[Service]
Delegate=memory pids cpu io yes

This issue has been discussed here top of kind project: https://github.com/kubernetes-sigs/kind/issues/2916 but the trick they propose will not work on macos !

Wait wait. I did a new test where I set as DOCKER_HOST the one pointing to the user ID

set DOCKER_HOST "unix:///run/user/501/podman/podman.sock"
❯ idp create \
              --color \
              --build-name my-konflux \
              --kind-config my-konflux-cfg.yaml \
              -p fork-konflux-ci/idp/dependencies \
              -p fork-konflux-ci/idp/konflux \
              -p fork-konflux-ci/idp/testing \
              --recreate
Nov  5 12:34:58 INFO Creating kind cluster logger=setup 
Nov  5 12:34:58 INFO Runtime detected logger=setup provider=podman 
########################### Our kind config ############################
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
    endpoint = ["https://gitea.cnoe.localtest.me"]
  [plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
    insecure_skip_verify = true
kind: Cluster
networking: {}
nodes:
- extraMounts:
  - containerPath: /var/lib/kubelet/config.json
    hostPath: /Users/cmoullia/.config/containers/auth.json
  extraPortMappings:
  - containerPort: 443
    hostPort: 8443
    protocol: TCP
  labels:
    ingress-ready: "true"
  role: control-plane

and that works. I will recreate a new VM on my macos and retest without any changes to the files as documented under kind rootless section to see what will happen

@nabuskey

cmoulliard commented 1 week ago

I did a new test where:

nabuskey commented 1 week ago

The original issue is already documented. I don't see any need to keep this issue open.