search

k3d-io / k3d

Little helper to run CNCF's k3s in Docker
https://k3d.io/
MIT License
5.46k stars 462 forks source link

[BUG] k3d registry create + cluster create --network host --registry use don't seem to work together #1418

Open akshaya-a opened 7 months ago

akshaya-a commented 7 months ago

What did you do

Following the docs,

k3d registry create... k3d cluster create --registry-use ...

works well together. While trying to debug a connectivity issue, I added --network host to the cluster create command. This fails to start the cluster due to:

Command '/usr/local/bin/k3d cluster create XYZ --kubeconfig-update-default=0 --image rancher/k3s:v1.25.3-k3s1 --wait --timeout=240s --registry-use ABC --network host' returned non-zero exit status 1. /usr/local/bin/k3d cluster create XYZ --kubeconfig-update-default=0 --image rancher/k3s:v1.25.3-k3s1 --wait --timeout=240s --registry-use ABC --network host b"\x1b[36mINFO\x1b[0m[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] Prep: Network \n\x1b[36mINFO\x1b[0m[0000] Re-using existing network 'host' (1da4a9f896cefba85d1d35694b84122b832079332322eea6d19796d528de2972) \n\x1b[36mINFO\x1b[0m[0000] Created image volume XYZ \n\x1b[36mINFO\x1b[0m[0000] Deleting cluster 'XYZ' \n" b"\x1b[33mWARN\x1b[0m[0000] Failed to connect to network 'host': Connection failed \n\x1b[33mWARN\x1b[0m[0000] Error response from daemon: container cannot be disconnected from host network or connected to host network \n\x1b[31mERRO\x1b[0m[0000] Failed Cluster Preparation: Failed to connect registry node 'ABC' to cluster network: Failed to connect to one or more networks \n\x1b[31mERRO\x1b[0m[0000] Failed to create cluster >>> Rolling Back \n\x1b[31mERRO\x1b[0m[0000] failed to get cluster: No nodes found for given cluster \n\x1b[31mFATA\x1b[0m[0000] Cluster creation FAILED, also FAILED to rollback changes! \n"

What did you expect to happen

k3d cluster create --network host respects previously created k3d registry create

Screenshots or terminal output

If applicable, add screenshots or terminal output (code block) to help explain your problem.

Which OS & Architecture

ubuntu x64

Which version of k3d

k3d version v5.6.0 k3s version v1.27.4-k3s1 (default)

Which version of docker

Client: Docker Engine - Community Version: 25.0.3 API version: 1.44 Go version: go1.21.6 Git commit: 4debf41 Built: Tue Feb 6 21:13:09 2024 OS/Arch: linux/amd64 Context: default

Server: Docker Engine - Community Engine: Version: 25.0.3 API version: 1.44 (minimum version 1.24) Go version: go1.21.6 Git commit: f417435 Built: Tue Feb 6 21:13:09 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.28 GitCommit: ae07eda36dd25f8a1b98dfbf587313b99c0190bb runc: Version: 1.1.12 GitCommit: v1.1.12-0-g51d5e94 docker-init: Version: 0.19.0 GitCommit: de40ad0

Client: Docker Engine - Community Version: 25.0.3 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.12.1 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.24.5 Path: /usr/libexec/docker/cli-plugins/docker-compose

Server: Containers: 2 Running: 1 Paused: 0 Stopped: 1 Images: 56 Server Version: 25.0.3 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb runc version: v1.1.12-0-g51d5e94 init version: de40ad0 Security Options: apparmor seccomp Profile: builtin cgroupns Kernel Version: 6.5.0-25-generic Operating System: Ubuntu 22.04.4 LTS OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 9.704GiB Name: xxx ID: db1d1237-4a37-49d5-a777-e17f0e010bec Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

akshaya-a commented 7 months ago

I should also add that I found the undocumented --default-network flag for registry create that seems pretty much what I want: https://github.com/k3d-io/k3d/blob/main/cmd/registry/registryCreate.go#L111

but then cluster create on use fails with

b"\x1b[36mINFO\x1b[0m[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] Prep: Network \n\x1b[36mINFO\x1b[0m[0000] Deleting cluster XYZ \n" b'\x1b[31mERRO\x1b[0m[0000] Failed Cluster Preparation: Failed Network Preparation: failed to create cluster network: failed to check for duplicate docker networks: failed to parse IP Prefix of network "host"\'s member <>: ParseAddr(""): unable to parse IP \n\x1b[31mERRO\x1b[0m[0000] Failed to create cluster