Closed doctorpangloss closed 1 year ago
@coutinhop PTAL
reproduces in 3.25 and containerd 1.7.0. Core issue is how the kube API client is constructed in Calico's libraries.
This will be fixed by https://github.com/projectcalico/calico/pull/7857 which will use inclusterconfig correctly on windows HPC
Closing this as fixed by https://github.com/projectcalico/calico/pull/7857 and https://github.com/tigera/operator/pull/2732
The
calico-windows-configmap
omitsKUBECONFIG: "C:\\CalicoWindows\\calico-kube-config"
. calico-node.exe api client doesn't look for the projected service token on Windows in the hostprocess environment, becauseconfig.ps1
always set KUBECONFIG.If it happens to be set to the node's joined kubeconfig, and mutual trust is used to join nodes, you will see the errors in this ticket.
If it isn't set, it will default to
C:\\CalicoWindows\\calico-kube-config
which will take precedence over the service account token.Thus the service account token is never used in windows hpc calico.
Adding
KUBECONFIG: "C:\\CalicoWindows\\calico-kube-config"
on machines with KUBECONFIG set resolves the issue, but only accidentally. Eventually the token in that file will expire.Related: https://github.com/projectcalico/calico/issues/7337 Might be related: https://github.com/projectcalico/calico/pull/5910
Expected Behavior
Calico for Windows hostprocess containers install method should use its projected service account correctly.
Current Behavior
Instead, when
KUBECONFIG
is set,node
uses the node's kubeconfig, which has a different user without access to Calico resources. This is unexpected.install
logs:node
logs:Observe that calico-node is using the authorization token in
c:\\k\\config
instead of thecalico-node
service account's token as specified.I believe this is because
node
is looking in the wrong path on containerd 1.7.0-beta.3Possible Solution
Use the right path to the service account token on 1.7.0-beta.3 and later.
Steps to Reproduce (for bugs)
kubeadm
workflow, and also how I connect Windows nodes to ak0s
cluster. This means you will use a bootstrapping kubeconfig for the node, then approve a CSR for the serving node. For example:On your machine, get bootstrapping kubeconfig
On Windows worker, install Kubelet
Set KUBECONFIG
Approve the csr
C:/k/config
is authorizingsystem:node:worker-hostname
. Mine in these logs isappmana-006
, a test machine.calico-node
.node
indicate it has authorized withsystem:node:appmana-006
. It should becalico-node
.clusterinformation
.KUBECONFIG: "C:\\CalicoWindows\\calico-kube-config"
to the configmap resolves the issue.Your Environment