Open lbergesio opened 1 month ago
I have seen that the problem is that the docker network was not created with --gateway
so, dockerNetworkInspect
returns info.gateway = nil
and then the StaticIP is not respected in func (d *Driver) Create() error
because to add it to params the check is:
} else if gateway != nil && staticIP != "" {
params.Network = networkName
params.IP = staticIP
}
The output of the network inspect is
{"Name": "leotest","Driver": "bridge","Subnet": "192.168.100.0/24","Gateway": "","MTU": 0, "ContainerIPs": []}
imho the code should check for the Subnet, check if the staticIp is free and use it.
I'm facing similar issue:
Minikube version: v1.33.1 OS: Ubuntu 22.04, AMD64 Docker version: 26.1.4
@ComradeProgrammer would you please take a look
It works for me, with below configuration
Now, i am changing and re applying the minikube with another static ip "192.168.100.25" and this also works.
Let me know if you have any questions.
It works for me, with below configuration
![]()
![]()
Now, i am changing and re applying the minikube with another static ip "192.168.100.25" and this also works.
Let me know if you have any questions.
It works if you specify the gateway, please check https://github.com/kubernetes/minikube/issues/19284#issuecomment-2236013065
/assign
Why would you use a static ip without a gateway? i get it for docker, but in Kubernetes you need the gateway for the nodes to communicate in a cluster. What would you expect to happen if you try and add nodes?
Why would you use a static ip without a gateway? i get it for docker, but in Kubernetes you need the gateway for the nodes to communicate in a cluster. What would you expect to happen if you try and add nodes?
I'm not sure I get your question, but why not? Afaik here you are configuring the docker network. The k8s nodes can use the static IP as default gw, but that's a different GW. In any case if I'm specifying the static IP and the network and that is not being respected I would expect the command to fail or at least warn about it
For my education - if you create a minikube control plane with a static IP, for example, I don't see how you can add any other nodes (how will the other nodes be allocated IP addresses?). I think you are correct about the error message,, "fixing it," by removing the gateway check - I suspect will cause other issues (Ingress etc.)
Again, not sure I get your point. But it is true that with current code you can not start a cluster with static-ip and >1 nodes, it fails with:
leo:~$ minikube start -p leo --network leo --static-ip 192.168.111.100 -n 2
....
👍 Starting worker node leo-m02 in cluster leo
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=3950MB) ...
❌ Exiting due to GUEST_START: failed to start node: adding node: Failed to start host: can't create with that IP, address already in use
But that imho is a bug in the code, when you do not run with a static ip it assignes proper ips to the nodes in the network range:
leo:~$ minikube start -p leo --network leo -n 2
....
👍 Starting worker node leo-m02 in cluster leo
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=3950MB) ...
🌐 Found network options:
▪ NO_PROXY=192.168.111.2
🐳 Preparing Kubernetes v1.28.3 on Docker 24.0.7 ...
▪ env NO_PROXY=192.168.111.2
🔎 Verifying Kubernetes components...
🏄 Done! kubectl is now configured to use "leo" cluster and "default" namespace by default
leo:~$ minikube ip -p leo -n leo
192.168.111.2
leo:~$ minikube ip -p leo -n leo-m02
192.168.111.3
So, it depends how you want to treat --static-ip
since that is used as the first node ip you can say that you can't use --static-ip
with -n
>1 or you can look for the next free ip in the network.
Regarding routing, the default route is
default via 192.168.111.1 dev eth0
But the docker network is a bridge network, you do not need an ip there, you can do:
leo:~$ minikube ssh -p leo
docker@leo:~$ ip route
default via 192.168.111.1 dev eth0
10.244.0.2 dev veth1990a1d1 scope host
10.244.1.0/24 via 192.168.111.3 dev eth0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.111.0/24 dev eth0 proto kernel scope link src 192.168.111.2
docker@leo:~$ sudo ip route del default via 192.168.111.1 dev eth0
docker@leo:~$ ip route
10.244.0.2 dev veth1990a1d1 scope host
10.244.1.0/24 via 192.168.111.3 dev eth0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.111.0/24 dev eth0 proto kernel scope link src 192.168.111.2
docker@leo:~$ sudo ip route add default dev eth0
docker@leo:~$ ip route
default dev eth0 scope link
10.244.0.2 dev veth1990a1d1 scope host
10.244.1.0/24 via 192.168.111.3 dev eth0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.111.0/24 dev eth0 proto kernel scope link src 192.168.111.2
All this was creating the docker network like this (no gateway):
docker network create --driver bridge --subnet=192.168.111.0/24 --opt com.docker.network.bridge.name=leo_br leo --gateway 192.168.111.1
leo:~$ ifconfig leo_br
leo_br: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.111.1 netmask 255.255.255.0 broadcast 192.168.111.255
ether 02:42:46:aa:c2:dd txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
But what I said about the routing is not needed anyway, if you do not specify the gateway when creating the docker network, still docker assigns an ip to the bridge interface, so minikube could retrieve the gateway ip from the system instead from docker network inspect
and it will find it:
leo:~$ docker network create --driver bridge --subnet=192.168.111.0/24 --opt com.docker.network.bridge.name=leo_br leo
83ac3fdb2b5a2c4272cec41ef72758711bcbd7d23497591944f060a3481662a5
leo:~$ ifconfig leo_br
leo_br: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.111.1 netmask 255.255.255.0 broadcast 192.168.111.255
ether 02:42:46:aa:c2:dd txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Thanks for taking the time to respond with all those details. - very much appreciated . I'm wondering what the initial intention was in adding the static ip option to Minikube in the first place. If you do as you suggest, retrieving IP addresses for additional nodes from the docker gateway - wouldn't that be a security issue for the cluster - as anything on the docker network (external to the Minikube cluster that is ) can access directly the nodes.?
I think you can already today even not using static-ip. Anyway, you can limit that with the appropriated up tables rules, don't know if any are added today.
What Happened?
I have a docker network previously created:
If I run the following command the ip is not repected:
But if I do not specify the network then, a new one is created and the ip is respected:
Attach the log file
log.txt
Operating System
Ubuntu
Driver
Docker