vexxhost / magnum-cluster-api

Cluster API driver for OpenStack Magnum
Apache License 2.0
45 stars 21 forks source link

fr: support setting kubelet --node-ip using workers' primary NIC #442

Open andrewbonney opened 3 days ago

andrewbonney commented 3 days ago

We are using secondary networking in some Kubernetes clusters via Multus. Unfortunately, when we attach additional Neutron ports to k8s worker VMs (or if bare metal nodes have additional NICs), the IP addresses for these are automatically added to the node's list of InternalIPs. As the secondary network is not universally available (for example it's not used by controller nodes), attaching these interfaces can break connectivity to pods' log output and the ability to run kubectl exec.

It would be helpful if --node-ip could be set for the kubelet at deploy time based on the IP it is allocated. I can't immediately think of a reason this would need to be restricted to this use case, so it may be possible to do so universally.

jrosser commented 3 days ago

@mnaser we discussed briefly on irc about adding something here https://github.com/vexxhost/magnum-cluster-api/blob/824fb57dd3496814e925752d15836c6db1e92cd6/magnum_cluster_api/resources.py#L727

But we are not sure how to identify the correct interface, because we see this in the openstackmachine status

kubectl -n magnum-system describe machine/kube-yxpub-default-worker-j8lnw-6fwzm-cgm4

<snip>
Status:
  Addresses:
    Address:        172.16.129.102
    Type:           InternalIP
    Address:        10.0.0.44
    Type:           InternalIP
    Address:        kube-yxpub-default-worker-2s4lc-s96zc
    Type:           InternalDNS

We need to either know up front what the node IP is, or have a way of distinguising the primary interface from any others which are present.