k0sproject / k0smotron

k0smotron
https://docs.k0smotron.io/
Other
424 stars 39 forks source link

NLLB setting is not compatible with CAPV #445

Open nekwar opened 5 months ago

nekwar commented 5 months ago

Problem summary

In order to create child cluster with controllers as VMs (not as pods) on vSphere with the help of CAPV, you need to create VsphereCluster resource:

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: VSphereCluster
metadata:
  name: '${CLUSTER_NAME}'
  namespace: '${NAMESPACE}'
spec:
  controlPlaneEndpoint:
    host: ${CONTROL_PLANE_ENDPOINT_IP}
    port: 6443
  identityRef:
    kind: Secret
    name: '${CLUSTER_NAME}'
  server: '${VSPHERE_SERVER}'
  thumbprint: '${VSPHERE_TLS_THUMBPRINT}'

which contains controlPlaneEndpoint parameter. This parameter later is being passed to k0s.yaml as spec.api.externalAddress option value. This behaviour is definitely expected.

But the thing is, when you're trying to enable Node-local Load Balancing using K0sControlPlane resource, the k0s cluster won't be created with the following message:

root@foobar-0:/home/foobar# k0s install controller --enable-worker --no-taints --config /etc/k0s.yaml
Error: invalid node config: spec: network.nodeLocalLoadBalancing.enabled: Forbidden: node-local load balancing cannot be used in conjunction with an external Kubernetes API server address

Expected behaviour

k0s should be installable with both externalAddress option for API and with NLLB enabled. The reason for that is that NLLB is used for cluster-internal communication, and it does not intersect with the ability to use load balancer in front of k0s API for it to be accessible externally.

nekwar commented 5 months ago

To be honest, I don't know what is the best place to put this issue to (here or k0s repo), so if it's not the right place, please tell me, I'll recreate the issue in k0s repo