search

armory / minnaker

Spinnaker on Lightweight Kubernetes (K3s)
Apache License 2.0
144 stars 53 forks source link

spinnaker-operator not resolving localhost correctly #85

Open mikael-lindstrom opened 3 years ago

mikael-lindstrom commented 3 years ago

When using multipass on OSX localhost inside k3s containers returns the OSX hosts IP instead of 127.0.0.1. This is due to the multipass vm's /etc/resolve.conf being used by CoreDNS and it contains search localdomain. Running nslookup localhost inside the spinnaker-operator container returns the OSX host IP which means that communication between the operator container and the halyard container fails.

As a workaround, before I ran ./scripts/install.sh -o I copied /etc/resolv.conf inside multipass to /home/ubuntu/resolv.conf, removed search localdomain and changed nameserver to 8.8.8.8, result looks like this:

nameserver 8.8.8.8
options edns0

I then updated this line and added K3S_RESOLV_CONF="/home/ubuntu/resolv.conf":

  curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--tls-san $(cat ${BASE_DIR}/secrets/public_ip)" INSTALL_K3S_VERSION="v1.19.7+k3s1" K3S_KUBECONFIG_MODE="644" K3S_RESOLV_CONF="/home/ubuntu/resolv.conf" sh -

After this I ran ./scripts/install.sh -o and it successfully launches Spinnaker.

away168 commented 3 years ago

Hmm, when I use Multipass on my OS X - I didn't have an issue with resolving minnaker's ip address.

Thanks for adding the K3S_RESOLV_CONF line. The other option we currently provide is to explicitly specify the ip address with the -p flag.

mikael-lindstrom commented 3 years ago

Tested with using the -P flag and it initially worked. However when I stopped and restarted the multipass VM the problem came back and the spinnaker-operator container fails with being unable to resolve localhost properly. I suspect this is related to multipass/network on my computer since it works for you and not related to minnaker.