Localhost is not respected by kubeadm for advertiseAddress

onorua commented 7 years ago

kubeadm config:

kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
api:
  advertiseAddress: "127.0.0.1"
networking:
  podSubnet: "10.244.0.0/16"
controllerManagerExtraArgs:
  horizontal-pod-autoscaler-use-rest-clients: "true"
  horizontal-pod-autoscaler-sync-period: "10s"
  node-monitor-grace-period: "10s"
apiServerExtraArgs:
  runtime-config: "api/all=true"
  feature-gates: "TaintBasedEvictions=true"
  admission-control: "NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds"
kubernetesVersion: "v1.8.0"

What happened?

In case I set advertiseAddress to 127.0.0.1, it doesn't respect that value, but rather discover it automatically. I can see that following code is doing that: https://github.com/kubernetes/apimachinery/blob/dc1f89aff9a7509782bde3b68824c8043a3e58cc/pkg/util/net/interface.go#L384

But localhost is completely valid use case. Let me elaborate, at the moment we have haproxy working as a API server load balancer, as well as health checker for kube-proxy.

What you expected to happen?

Respect configured value.

How to reproduce it (as minimally and precisely as possible)?

take kubeadm config mentioned above, executed:

kubeadm init --config kubeadm.conf --skip-preflight-checks --dry-run

Check for api server manifest, all the config maps, basically everything pointing to the automatically discovered IP, instead one from configuration file.

luxas commented 7 years ago

Why wouldn't you like this to be a normal IP?

onorua commented 7 years ago

because that is the only IP available on each and every host, this gives broad possibilities to use client side load balancers to reach API server. You can check HA for kubeadm document, there is a definition of "well known IP", which is not defined yet, localhost is actually a well know, and makes it simple to use client side LB as well as iptables proxying.

hbasappa commented 7 years ago

hitting the same issue..would love the support for 127.0.0.1/localhost in a closed system

fejta-bot commented 6 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

onorua commented 6 years ago

@luxas we have an issue here, according to conversation at my PR #57607 people are not convinced that there is a reasonable to expose api server as localhos, and suggest to make changes anywhere else (meaning kubeadm). So, what do you propose here? As my previous PRs were rejected due to logical reasoning, I believe we should come to consensus before implementing it. Do you actually want support for client side load balancer (have support for any environment, not only cloud providers) or no?

fejta-bot commented 6 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten /remove-lifecycle stale

timothysc commented 6 years ago

/cc @fabriziopandini This could affect some HA stories where they have local proxies.

jakolehm commented 6 years ago

We are also hitting into this.. we need client side load balancing and this kinda makes it really hard (workaround is to update cluster-info configmap).

SpComb commented 6 years ago

The problematic utilnet.ChooseBindAddress only seems to be called in two places: with the kubeapiserver --bind-address=... value and then with the kubeadm --apiserver-advertise-address=... value.

However, for the kubeapiserver case, it's only called in the case of kubeapiserver --advertise-address= or --advertise-address=0.0.0.0. Specifically, it looks like kubeapiserver --advertise-address=127.0.0.1 will actually get respected, and it will NOT use the unfortunately-named ChooseBindAddress to convert any --insecure-bind-address=127.0.0.1 value into an AdvertiseAddress based on the IP addresses configured on the host.

I think it's a bug for kubeadm to handle an explicit kubeadm init --apiserver-advertise-address=127.0.0.1 as if you were using --apiserver-advertise-address=0.0.0.0? The --advertise-address is not the same thing as the --[insecure-]bind-address, and the apiserver never calls utilnet.ChooseBindAddress with the --advertise-address value?

fejta-bot commented 6 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

neolit123 commented 6 years ago

127.0.0.1 should probably not be allowed for an advert. address for the api-server.

i concur with the associated PR being closed: https://github.com/kubernetes/kubernetes/pull/57607

kubernetes / kubeadm