search

kubernetes-sigs / sig-windows-tools

Repository for tools and artifacts related to the sig-windows charter in Kubernetes. Scripts to assist kubeadm and wincat and flannel will be hosted here.
Apache License 2.0
125 stars 122 forks source link

Which sandbox (pause) image to use #323

Closed Mik4sa closed 1 year ago

Mik4sa commented 1 year ago

In this kubernetes guide to configure containerd as the CRI runtime there is the following section:

Overriding the sandbox (pause) image

In your containerd config you can overwrite the sandbox image by setting the following config:

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "registry.k8s.io/pause:3.2"

You might need to restart containerd as well once you've updated the config file: systemctl restart containerd.

Please note, that it is a best practice for kubelet to declare the matching pod-infra-container-image. If not configured, kubelet may attempt to garbage collect the pause image. There is ongoing work in containerd to pin the pause image and not require this setting on kubelet any longer.

Currently our guide has no such section to inform about this. As far as I could see by just following the guide (as I did) we end with two different configured images. For containerD 1.7.1 the default configuration uses registry.k8s.io/pause:3.8 In our PrepareNode.ps1 script we use mcr.microsoft.com/oss/kubernetes/pause:3.6: https://github.com/kubernetes-sigs/sig-windows-tools/blob/d744f5488dd350fa967b3920d698b268e214e14d/hostprocess/PrepareNode.ps1#L66

This raises several questions:

jsturtevant commented 1 year ago
  • Which would be the right repo/image for windows? registry.k8s.io/pause or mcr.microsoft.com/oss/kubernetes/pause

they are both built from the same source. The difference is that the mcr.microsoft.com version has binaries signed by Microsoft. We typically reference the registry.k8s.io registries in the kuberentes repos.

jsturtevant commented 1 year ago

I am not certian but I don't think '--pod-infra-container-image=mcr.microsoft.com/oss/kubernetes/pause:3.6 is used any more, the containerd configuration is what is used when deployed with containerd

I don't think it needs to be configured (custom containerd configurations are out of scope IMO) and should be the default that containerd version ships with.

Mik4sa commented 1 year ago

I am not certian but I don't think '--pod-infra-container-image=mcr.microsoft.com/oss/kubernetes/pause:3.6 is used any more, the containerd configuration is what is used when deployed with containerd

Looks like it's still in use See here: https://github.com/kubernetes/kubernetes/blob/7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647/pkg/kubelet/config/flags.go#L53-L54 and here: https://github.com/kubernetes/kubernetes/blob/7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647/cmd/kubelet/app/options/container_runtime.go#L25-L26

I don't think it needs to be configured (custom containerd configurations are out of scope IMO) and should be the default that containerd version ships with.

Ok, I'm fine with it. I'll then just change it manually for my clusters.

jsturtevant commented 1 year ago

will be removed in a future release. Image garbage collector will get sandbox image information from CRI.

We should probably remove it since it is depreciated. That way we are proactive on it.

Mik4sa commented 1 year ago

I'm unsure, but I think we can't remove it right now: https://github.com/kubernetes/kubernetes/pull/118544