iwilltry42
commented
4 years ago Just leaving some notes here:
- By default, k3s runs
flannelas the CNI plugin invxlanbackend mode - flannel does not run on top of Kubernetes, as one might be used to, but rather it's included in k3s itself: https://github.com/rancher/k3s/tree/master/pkg/agent/flannel
- on the k3s nodes (containers in the case of k3d), we'll get routes like
10.42.2.0/24 via 10.42.2.0 dev flannel.1 onlinkfor routing between podCIDRs assigned to the nodes
dcharbonnier
What did you do? Does K3D support having worker nodes running on separate servers?
I run k3d create on 192.168.0.104 (node1) I run k3d add-node --k3s https://192.168.0.104:6443 --k3s_token=mytoken on 192.168.0.105 (node2)
I can kubectl with no issue. pods are created on both nodes. I can exec "bash" on containers running on either node, no problem.
However, containers on node1 don't seem to have network connectivity to containers on node2, nor the other way around. I can manually pick apart the pieces and see it not working (exec bash and ping). But, it's easiest to see by creating a deployment/service/ingress and forcing the deployment to node1. Since traefik also runs on node1, I can get to the service without issue. When I force the deployment to node2, traefik can't get there.
How was the cluster created? k3d create -x --token=12345678901234567890 -x --advertise-address=192.168.0.104 -p 80:80 -p 443:443
What did you do afterwards? on 192.168.0.105: k3d add-node --k3s https://192.168.0.104:6443 --k3s_token=12345678901234567890
What did you expect to happen? pods created on node1 have IPs like 10.42.0.x pods created on node2 have IPs like 10.42.1.x
when creating a busybox pod on node1 and attaching to a shell there, I cannot ping pods on node2. nor the other way around. I expected to be able to.
Which OS & Architecture? Linux x86 for both machines
Which version of
k3d? v1.5.1Which version of docker? 19.03.3