search

kubernetes-sigs / cluster-api-provider-openstack

Cluster API implementation for OpenStack
https://cluster-api-openstack.sigs.k8s.io/
Apache License 2.0
286 stars 252 forks source link

Upgrading clusters without Octavia #912

Closed petrutlucian94 closed 3 years ago

petrutlucian94 commented 3 years ago

/kind bug

What steps did you take and what happened:

I've tried upgrading k8s using the following command:

kubectl patch kubeadmcontrolplane control-plane --type=json \
  -p '[{"op": "replace", "path": "/spec/version", "value": "1.18.12"}]'

We didn't have Octavia installed, so what happened was that the floating IP moved to the new instance, which was trying to join the existing cluster using the floating ip that had already been moved.

What did you expect to happen:

I was expecting the k8s upgrade operation to complete successfully.

Anything else you would like to add:

I'm not sure if this can be considered a bug or just an undocumented limitation. Is there a way in which we could get the cluster upgrade operation to work without having Octavia (Neutron LBaaS)? One possible approach would be to use the private IPs instead of the floating IP as part of the upgrade workflow.

Environment:

hidekazuna commented 3 years ago

AFAIK, this is an undocumented limitation.

sbueringer commented 3 years ago

@petrutlucian94 I don't think there is a way to do this. Core CAPI is upgrading by joining new control plane nodes, without any kind of loadbalancing mechanism between control planes nodes this simply doesn't work. Right now loadbalancing is only implemented based on Octavaia. There are probably other ways to achieve this without loadbalancers, but not sure how and there's currently no alternate implementation in CAPO.

petrutlucian94 commented 3 years ago

We've managed to achieve this using kube-vip as an alternative load balancer. Worth mentioning that if port security is enabled, the AllowedAddressPairs option must be used in order to enable VIP traffic. Luckily, a recent CAPO change added this feature: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/commit/d32407da0da0171938b71d6d1c6147308ef5a70b

chess-knight commented 5 months ago

Hi @petrutlucian94. I am utilizing kube-vip for the HA control-plane on OpenStack using CAPO. I will be grateful if you can share minimal example (manual steps for OpenStack and CAPO manifests) of how to do that. If I understand correctly, I need to create a network and port with a fixed IP beforehand and then somehow configure CAPO, but I have not been successful so far. Thanks