search

amimof / kubernetes-the-right-way

Install Kubernetes with Ansible
MIT License
28 stars 7 forks source link

kube-proxy on master nodes? #51

Closed anton-johansson closed 4 years ago

anton-johansson commented 5 years ago

I'm trying to install the metrics server into my cluster. It requires you to add an APIService that registers itself as an API extension in the API server. However, my masters needs to be able to access this using a Service clusterIP, which it currently cannot, so the APIService fails.

Reading around a bit: https://github.com/kubernetes/kubernetes/issues/66231

It looks like people install kube-proxy on the masters to achieve this, but it feels a bit weird.

Have you got any idea on how to do this best with Kubernetes The Right Way? There is also a discussion here where they recommend adding an additional API server as a pod inside the cluster, but I'm not quite sure...

EDIT: Running kube-proxy on masters feels really odd. It's not something that I'd want to do.

anton-johansson commented 5 years ago

Related: kubernetes-incubator/metrics-server#281

amimof commented 5 years ago

I feel that I need to do some research and testing to understand this completely. But I agree that running kube-proxy on the apiserver feels strange. Feels that the implementation of metrics-server could be different.

anton-johansson commented 4 years ago

I managed to get the metrics-server up and running. I got a bit of help in the avove issue.

Short summary: To use APIService extensions to the API server when you are not running kube-proxy on your master nodes, you must set the flag --enable-aggregator-routing=true on kube-apiserver. This effectively means that the API server won't use the service IP's to reach the APIService. It will instead use one of the endpoints of the service. Now, this won't work out of the box either, because the API server cannot reach Pod IP's either. But if you set metrics-server (or whatever extension you are adding) to use hostNetwork: true, it will work (of course with the downside of using hostNetwork, but I think I can live with that).