Cluster creation failed with vexxhost images

[mathgonzalezz]

Im seeing the creation is not working with the vexxhost hosted images

ubuntu-v1.26.7: ERROR kubeletversion: the kubelet version is higher than the control plane version. this is not a supported version skew and may lead to a malfuctional cluster. kubelet version: : "1.26.7" control plane version: "1.25.3"

and in the capi logs i see: E0405 21:44:59.656711 1 controller.go:329] "Reconciler error" err="failed to create cluster accessor: error creating http client and mapper for remote cluster \"magnum-system/kube-dlcgr\": error creating client for remote cluster \"magnum-system/kube-dlcgr\": error getting rest mapping: failed to get API group resources: unable to retrieve the complete list of server APIs: v1: Get \"https://192.168.60.117:6443/api/v1?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" controller="machine" controllerGroup="cluster.x-k8s.io" controllerKind="Machine" Machine="magnum-system/kube-dlcgr-mh5d6-ssq95" namespace="magnum-system" name="kube-dlcgr-mh5d6-ssq95" reconcileID="cbdafd7f-eb57-4e33-ab81-213a13866712"

ubuntu-v.1.27.4: This version of kubeadm only supports deploying clusters with the control plane version >= 1.26.0. Current version: v1.25.3

capi cluster: "Reconciler error" err="failed to create cluster accessor: error creating http client and mapper for remote cluster \"magnum-system/kube-81xpf\": error creating client for remote cluster \"magnum-system/kube-81xpf\": error getting rest mapping: failed to get API group resources: unable to retrieve the complete list of server APIs: v1: Get \"https://192.168.60.45:6443/api/v1?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" controller="machine" controllerGroup="cluster.x-k8s.io" controllerKind="Machine" Machine="magnum-system/kube-81xpf-bxwdc-hjmlk" namespace="magnum-system" name="kube-81xpf-bxwdc-hjmlk" reconcileID="c4b1f77b-d89c-4935-94ce-9a015136d1d1"

Thanks!

[mnaser]

Can you share the cluster template a the cluster creation command you're using?

[mathgonzalezz]

Heres the template craetion:

openstack coe cluster template create \ --image ubuntu-2204-kube-v1.27.4 \ --external-network external \ --dns-nameserver ${dns} \ --master-lb-enabled \ --master-flavor m2_medium \ --flavor m2_medium \ --network-driver calico \ --docker-storage-driver overlay2 \ --coe kubernetes \ --label kube_tag=${version} \ k8s-${version}

And this is the cluster creation

openstack coe cluster create --cluster-template k8s-v1.72.4 --master-count 1 --node-count 1 --keypair osp testing127 --labels auto_scaling_enabled=true,min_node_count=1,max_node_count=5

Thanks

[mnaser]

Are you sure the images you're using are using the same version? It seems the errors seem to say that the images have a different version of Kubernetes than the one you are trying to deploy.

ERROR kubeletversion: the kubelet version is higher than the control plane version. this is not  a supported version skew and may lead to a malfuctional cluster. kubelet version: : "1.26.7" control plane version: "1.25.3"

I think you are trying to use 1.26.7 image with kube_tag set to 1.25.3.

[mathgonzalezz]

Are you sure the images you're using are using the same version? It seems the errors seem to say that the images have a different version of Kubernetes than the one you are trying to deploy.

ERROR kubeletversion: the kubelet version is higher than the control plane version. this is not  a supported version skew and may lead to a malfuctional cluster. kubelet version: : "1.26.7" control plane version: "1.25.3"

I think you are trying to use 1.26.7 image with kube_tag set to 1.25.3.

Yes because i only use one image and one template, the download is scripted with variables. But if you want i can re test again and share results

[mnaser]

You will need to download or build an image that matches the kube_tag, it is not possible to have a mismatch.

[mathgonzlez]

You will need to download or build an image that matches the kube_tag, it is not possible to have a mismatch.

Heres the command i use for download , create image and cluster template. export OS_DISTRO=ubuntu # you can change this to "flatcar" if you want to use Flatcar for version in v1.27.4; do \ [[ "${OS_DISTRO}" == "ubuntu" ]] && IMAGE_NAME="ubuntu-2204-kube-${version}" || IMAGE_NAME="flatcar-``kube-${version}"; \ #curl -LO https://object-storage.public.mtl1.vexxhost.net/swift/v1/a91f106f55e64246babde7402c21b87a/magnum-``capi/${IMAGE_NAME}.qcow2; \ openstack image create ${IMAGE_NAME} --disk-format=qcow2 --container-format=bare --property os_distro=${OS_DISTRO} --file=${IMAGE_NAME}.qcow2; \ openstack coe cluster template create \ --image $(openstack image show ${IMAGE_NAME} -c id -f value) \ --external-network external \ --dns-nameserver 192.168.40.245 \ --master-lb-enabled \ --master-flavor m2_large \ --flavor m2_large \ --network-driver calico \ --docker-storage-driver overlay2 \ --coe kubernetes \ --label kube_tag=${version} \ k8s-${version}; done;

And heres the cluster creation command: openstack coe cluster create --cluster-template k8s-v1.27.4 --master-count 1 --node-count 1 --keypair osp testing127 --labels auto_scaling_enabled=true,min_node_count=1,max_node_count=5,octavia_provider=ovn

But i still receiving the error of missmatch version: This version of kubeadm only supports deploying clusters with the control plane version >= 1.26.0 Current version: v1.25.3

Thanks

[mathgonzlez]

Hi, any news about this?

Thanks!

[mnaser]

@mathgonzalezz I think your issue is missing --merge-labels :)