intel / kubevirt-tdx

TDX Enabling for Kubevirt
Apache License 2.0
4 stars 1 forks source link

Appears to be incompatible with K8s 1.27 #1

Open igordcard opened 1 year ago

igordcard commented 1 year ago

The same exact installation steps were followed to install kubeadm 1.26 and kubeadm 1.27:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
sudo mkdir /etc/apt/keyring
sudo chmod 755 /etc/apt/keyrings
sudo curl -fsSLo /etc/apt/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-mark unhold kubelet kubeadm kubectl
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

sudo kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock # --v=5
sudo kubeadm init --pod-network-cidr=192.168.0.0/16 --cri-socket unix:///var/run/cri-dockerd.sock
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
echo "export KUBECONFIG=$HOME/.kube/config" >> ~/.bashrc
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/tigera-operator.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/custom-resources.yaml
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl get nodes -o wide
sudo systemctl restart kubelet

Except for downgrading Kubernetes to 1.26 in one of the attempts:

sudo apt-get install -qy --allow-downgrades --allow-change-held-packages kubelet=1.26.4-00 kubectl=1.26.4-00 kubeadm=1.26.4-00
sudo kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock
# follow rest of steps

And the KubeVirt part:

export VERSION=`v0.59.0`
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml
sleep 30
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml

With 1.27, it doesn't seem compatible with KubeVirt (both v0.59.0 and latest alpha v1.0.0-alpha.0): won't accept the CRD defined in kubevirt-operator.yaml:

$ kubectl create -f http://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml
error: resource mapping not found for name: "kubevirt" namespace: "kubevirt" from "http://github.com/kubevirt/kubevirt/releases/download/v0.59.0/kubevirt-cr.yaml": no matches for kind "KubeVirt" in version "kubevirt.io/v1"
ensure CRDs are installed first

Tested with upstream kubevirt, and assuming a similar situation for kubevirt-tdx. This is not critical in any way, just reporting here in the issues list as a heads-up for later.

LeiZhou-97 commented 1 year ago

Thanks. Sorry for the late reply. @igordcard

I only validate on k8s 1.25 and 1.26. In our env, we do not have 1.27.

Currently, our linux tdx stack do not support rhel9/centosstream9 (the upstream base image). So I cannot rebase to the latest upstream code (v1.0).