search

k3s-io / k3s

Lightweight Kubernetes
https://k3s.io
Apache License 2.0
28.1k stars 2.35k forks source link

Internal Cluster IP address being sent back to rancher instead of LAN Ip address #8600

Closed wryates closed 1 year ago

wryates commented 1 year ago

Environmental Info: K3s Version: v1.27.6+k3s1

Node(s) CPU architecture, OS, and Version:

Linux webhost1 5.15.0-86-generic #96-Ubuntu SMP Wed Sep 20 08:23:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: Rancher Server, 1 K3s Master

Describe the bug:

When registering cluster in rancher, k3s is not sending back the treafik external ip address

Steps To Reproduce:

Install Rancher on Docker on machine 1: docker run -d --restart=unless-stopped -p 80:80 -p 443:443 -v /opt/rancher:/var/lib/rancher --privileged rancher/rancher:latest --acme-domain example.com

Install k3s on machine 2: curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644 --cluster-init --node-external-ip <internal 192.168. address>

In Rancher, go to import an existing cluster and get the registration script

On the K3s Master I run the registration Script: kubectl apply -f https://example.com/v3/import/regname.yaml

When registering the cluster I get a Cluster Agent is not Connected, Waiting for API to be available message

Looking into it further, I found that K3s is reporting the internal Cluster IP address 10.43.0.1 instead of the LAN IP address which would be 192.168..

Expected behavior:

K3s should report the extneral IP address on the treafik service because rancher cant reach an internal kubernetes IP address on another machine

Actual behavior:

This is from the Cluster Management -> “Cluster Name” -> Related Resources -> “Mgmt Cluster” k3s1

The 10. address can not be hit because it's not on the actual k3s cluster, it's on a separate machine. It should be a 192.168..* address which would be the LAN IP address.

Additional context / logs:

Running: kubectl get all --all-namespaces gives me a list of the services and the IPs k3s2

The Ip address that should be sent to rancher to connect, should be the 192.168 external ip address for traefik, but its sending the kubernetes cluster-ip address

I have tried a lot of different flags: such as --bind-address,--advertise-address, --node-ip, and --node-external-ip with no luck.

I want to be able to keep the internal kubernetes cluster IP address, but report the external-ip address to rancher, so that i could access it

*IP ADDRESSES and URLs Removed for security

wryates commented 1 year ago

I ended up taking out the url domain and just running it locally, that worked and k3s was able to register. I got Node Proxy Manager in front, even tho all the machines have internet access. Ill keep digging

wryates commented 1 year ago

This is some kind of networking cert thing, going to close for now, the issue is different than i thought it was

wryates commented 1 year ago

So... all that.. just to make sure websockets were turned on... when you setup rancher, look at the console of your browser for websocket errors, I was getting them, once i fixed my node proxy manager, everything started working