replicatedhq / kURL

Production-grade, airgapped Kubernetes installer combining upstream k8s with overlays and popular components
https://kurl.sh
Apache License 2.0
741 stars 79 forks source link

Changing IP address #1613

Open evilmonkeyslayer opened 3 years ago

evilmonkeyslayer commented 3 years ago

Is it possible to change the ip address post airgap install?

For example, if I install kurl using a lan ip of 192.168.1.10 and then change the ip to 172.16.1.10. (if I want to move something from a build environment to a production environment for example)

Is there a process for this?

areed commented 3 years ago

It's possible with a single node cluster using these commands https://github.com/kubernetes/kubeadm/issues/338#issuecomment-407383077 as long as you're not using rook/ceph for storage.

Another option is to install docker first and then use 172.17.0.1 as the private IP. That will remain the IP of the docker0 interface even if the host IP changes.

evilmonkeyslayer commented 3 years ago

It's possible with a single node cluster using these commands kubernetes/kubeadm#338 (comment) as long as you're not using rook/ceph for storage.

Another option is to install docker first and then use 172.17.0.1 as the private IP. That will remain the IP of the docker0 interface even if the host IP changes.

Thanks @areed, just had a thought... (and I'll test this later..)

What happens if I simply either add a second IP (to the interface) which is the prod environment IP before doing the airgap install? Or creating a second vnic with the prod ip address? Once it's shifted over to the prod environment we remove the build ip but it still has its prod ip?

That way, when it comes to possible future clusters it should work right? Instead of being limited to single node. Or does it only pick up the first IP address on install?

areed commented 3 years ago

That might work. You'd probably need to specify the prod IP as the private address in the kurl spec:

  kurl:
    privateAddress: <prod IP>
evilmonkeyslayer commented 3 years ago

@areed Does changing the hostname have any impact upon this too? Gotten this mostly working by creating a dummy interface with nmcli that's set to autoconnect no. Then safely shut it down and convert it into a vsphere template, it gets renamed (hostname change) and its main interface gets changed to the ip address that was in use on the dummy interface.

For some reason when I run kubectl get nodes it's showing a status of "Ready,SchedulingDisabled". I can in theory manually run uncordon against it, but wondering why this occurs and would prefer to avoid it happening in the first place.

Thanks for the privateAddress bit btw.

Definitely appears changing the hostname breaks this. Just thinking is there a way to work around this? For example, running /opt/ekco/shutdown.sh:

/opt/ekco/shutdown.sh
Error from server (NotFound): nodes "newhostname" not found
pod "prometheus-k8s-1" deleted
pod "awx-postgres-0" deleted
Waiting for Ceph block devices to unmount
Waiting for Ceph block devices to unmount
Waiting for Ceph block devices to unmount
Waiting for Ceph block devices to unmount
areed commented 3 years ago

The /opt/ekco/shutdown.sh script runs automatically and cordons the node whenever it shuts down cleanly. The /opt/ekco/start.sh script uncordons the no when it restarts. The changed hostname on startup must be preventing the uncordon from completing successfully. You can confirm with journalctl -u ekco-reboot.

I found in the past when doing something similar with AMIs that I had to run hostnamectl to change the hostname back to match what it was when the image was created.

MikaelSmith commented 3 years ago

We use https://gist.github.com/MikaelSmith/2603b9c09079f02fc914f7860603f652 for a single node, which is specific to containerd but shouldn't be too hard to adapt for Docker if needed. Not sure how well it would work with multiple nodes.