Fail to create a kind cluster with Rootless nerdctl: unsupported restart policy "on-failure"

[jizusun]

What happened:

(devbox) ➜  KIND_EXPERIMENTAL_PROVIDER=nerdctl kind create cluster
ignoring unknown value "nerdctl" for KIND_EXPERIMENTAL_PROVIDER
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.29.2) 🖼 
 ✗ Preparing nodes 📦  
ERROR: failed to create cluster: command "docker run --name kind-control-plane --hostname kind-control-plane --label io.x-k8s.kind.role=control-plane --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --detach --tty --label io.x-k8s.kind.cluster=kind --net kind --restart=on-failure:1 --init=false --cgroupns=private --device /dev/fuse --publish=127.0.0.1:32769:6443/TCP -e KUBECONFIG=/etc/kubernetes/admin.conf kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245" failed with error: exit status 1
Command Output: time="2024-04-07T16:10:44+08:00" level=fatal msg="unsupported restart policy \"on-failure\", supported policies are: [\"always\"]"

What you expected to happen:

Successfully create a kind cluster as described here: https://kind.sigs.k8s.io/docs/user/rootless/#creating-a-kind-cluster-with-rootless-nerdctl

How to reproduce it (as minimally and precisely as possible):

1. Setup the rootless containerd and nerdctl

   
   (devbox) ➜ nerdctl version
   WARN[0000] unable to determine buildctl version: exec: "buildctl": executable file not found in $PATH 
   Client:
   Version:       v1.7.4
   OS/Arch:       linux/amd64
   Git commit:    7b5f7e0d8f705ed4e54f7040512327e231433366
   buildctl:
   Version:

Server: containerd: Version: v1.6.23 GitCommit: b69f1ad231b6d87eeb30504398075a92d615e83e runc: Version: 1.1.7-0ubuntu1~22.04.2 (devbox) ➜ swf-infra-as-code git:(sealos-single) ✗

3. Create a kind cluster according to here: https://kind.sigs.k8s.io/docs/user/rootless/#creating-a-kind-cluster-with-rootless-nerdctl

**Anything else we need to know?**:

Rootless nerdctl works okay for me

(devbox) ➜ docker run -d nginx
1344fc38be2c24b0965632949410dc4fc3481f04a3049b76408d5510484a0622 (devbox) ➜ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1344fc38be2c docker.io/library/nginx:latest "/docker-entrypoint.…" 6 seconds ago Up nginx-1344f (devbox) ➜ ls -lah $(which docker) lrwxrwxrwx 1 root root 22 Mar 13 12:32 /usr/local/bin/docker -> /usr/local/bin/nerdctl

**Environment:**

- kind version: (use `kind version`): `kind v0.22.0 go1.22.1 linux/amd64`
- Runtime info: (use `docker info`, `podman info` or `nerdctl info`):

(devbox) ➜ nerdctl info Client: Namespace: default Debug Mode: false

Server: Server Version: v1.6.23 Storage Driver: overlayfs Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Log: fluentd journald json-file syslog Storage: native overlayfs Security Options: apparmor seccomp Profile: builtin cgroupns rootless Kernel Version: 5.15.0-83-generic Operating System: Ubuntu 22.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 32 Total Memory: 47.13GiB Name: node1 ID: f918bc52-745e-4bed-92c1-8afcf8372732

WARNING: No cpuset support

- OS (e.g. from `/etc/os-release`):

➜ cat /etc/*release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS" PRETTY_NAME="Ubuntu 22.04.2 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.2 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy

- Kubernetes version: (use `kubectl version`):
- Any proxies or other special environment settings?:

```[tasklist]
### Tasks

[jizusun]

Hi @yankay I see you're very experienced about this https://github.com/kubernetes-sigs/kind/actions/workflows/nerdctl.yaml Can you guide me a little bit?

[aojea]

/assign @estesp @AkihiroSuda @yankay

[k8s-ci-robot]

@aojea: GitHub didn't allow me to assign the following users: estesp.

Note that only kubernetes-sigs members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see the contributor guide

In response to [this](https://github.com/kubernetes-sigs/kind/issues/3571#issuecomment-2041433641): >/assign @estesp @AkihiroSuda @yankay Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[AkihiroSuda]

Server: containerd: Version: v1.6.23

on-failure needs containerd v1.7 or later https://github.com/containerd/containerd/commit/3df7674058301f290fc148719d483e47f4118494

[AkihiroSuda]

/kind question

[k8s-ci-robot]

@AkihiroSuda: The label(s) kind/question cannot be applied, because the repository doesn't have them.

In response to [this](https://github.com/kubernetes-sigs/kind/issues/3571#issuecomment-2041436065): >/kind question > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[aojea]

@jizusun please use containerd v1.7+