containerd image registry configurations are deprecated

AndiDog commented 1 year ago

Both registry mirrors and credentials configurations are affected

https://github.com/containerd/containerd/blob/main/docs/cri/registry.md says

registry.mirrors and registry.configs as described in this document have been DEPRECATED. As described in the cri config you should now use the form
[plugins."io.containerd.grpc.v1.cri".registry]
   config_path = "/etc/containerd/certs.d"
NOTE: registry.configs.*.auth is DEPRECATED and will NOT have an equivalent way to store unecrypted secrets in the host configuration files. However, it will not be removed until a suitable secret management alternative is available as a plugin. It > remains supported in 1.x releases, including the 1.6 LTS release.

Research how mirrors should be specified now. We likely need the server/host fields.
Investigate availability of secret management for containerd
Check if the feature gets dropped, and from which version the deprecation applies
Update cluster-XXX apps

bdehri commented 1 year ago

According to the documentation, there is no way of passing unencrypted credentials for now. It is still supported with 1.6 LTS release. IMO, we can keep our config as it is until support for unencrypted auth token implemented with the new interface.

alex-dabija commented 1 year ago

Team Turtles owns containerd.

njuettner commented 1 year ago

For reference: https://github.com/containerd/containerd/blob/main/docs/cri/registry.md#configure-registry-credentials

primeroz commented 1 year ago

Current status

Registry Mirrors

Configuration of mirrors is already supported in CRI using the host.toml

Registry Authentication

Even though is already deprecated there is no proper secrets-management plugin alternative available as far as i can tell.

One possible solution, as pointed out here and here, is to use headers in the host.toml

$ echo -n "${USERNAME}:${PASSWORD}" | base64
dXNlcjpwYXNzd29yZA==

hosts.toml
server = "https://xx.xx.xx/"
[host."https://xx.xx.xx"]
capabilities = ["pull", "resolve", "push"]
[host."https://xx.xx.xx".header]
authorization = "Basic xxxxxxxxxxx"

I am not sure if this is worth doing since

1.6 ( LTS ) supports the current configuration format
- It also seems supported on 1.7 which is the last major release of containerd before 2.x
- What if flatcar / ubuntu / capi-image builder updates the version of Containerd to one that does not support the auth configurtation ?
Eventually before removing the feature completely there will be a proper alternative
It does seem in some cases, especially for dynamic credentials, containerd might not implement a solution and rather rely , at least in k8s, on https://kubernetes.io/docs/tasks/administer-cluster/kubelet-credential-provider/

primeroz commented 1 year ago

as decided in standup we should wait for the proper solution to the credentials deprecation before moving on with this.

moving to blocked

Gacko commented 3 months ago

IIRC @giantswarm/team-honeybadger recently worked on the registry mirrors while implementing Zot. Do you think this also solves this issue?

AndiDog commented 3 months ago

We still offer a way in the cluster chart to use the outdated config:

[plugins."io.containerd.grpc.v1.cri".registry.configs."{{$value.endpoint}}".auth]

However the above isn't used for default values.

And we're also not using the recommended config_path = ... style.

So let's keep this open. We'll definitely notice once containerd makes breaking changes.

giantswarm / roadmap