crio 1.23.4 create container error on centos 7

[hanfengyizu]

What happened?

When I install crio 1.23.4 from source code on centos 7. For installing kubelet 1.23.3. I`m unable to find any container when I excute command 'crictl ps'. The crio log as follow:

Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.640338270+08:00" level=debug msg="Setup seccomp from profile field: &SecurityProfile{ProfileType:RuntimeDefault,LocalhostRef:,}" file="seccomp/seccomp.go:194" id=24eca757-1497-489c-8356-089cc43529ed name=/runtime.v1.RuntimeService/CreateContainer
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.640392713+08:00" level=debug msg="Setting container's log_path = /var/log/pods/kube-system_etcd-10-0-135-50-vm10-0-135-50_dca91be4d9d4850f3ca0a2251805edd7, sbox.logdir = etcd-10-0-135-50/0.log, ctr.logfile = /var/log/pods/kube-system_etcd-10-0-135-50-vm10-0-135-50_dca91be4d9d4850f3ca0a2251805edd7/etcd-10-0-135-50/0.log" file="container/container.go:414"
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.640473925+08:00" level=debug msg="CONTAINER USER: 0" file="server/container_create.go:214" id=24eca757-1497-489c-8356-089cc43529ed name=/runtime.v1.RuntimeService/CreateContainer
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.640874220+08:00" level=debug msg="/etc/system-fips does not exist on host, not mounting FIPS mode subscription" file="subscriptions/subscriptions.go:198"
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.641747618+08:00" level=debug msg="running conmon: /usr/local/bin/conmon" args="[-b /run/containers/storage/overlay-containers/0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c/userdata -c 0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c --exit-dir /var/run/crio/exits -l /var/log/pods/kube-system_etcd-10-0-135-50-vm10-0-135-50_dca91be4d9d4850f3ca0a2251805edd7/etcd-10-0-135-50/0.log --log-level debug -n k8s_etcd-10-0-135-50_etcd-10-0-135-50-vm10-0-135-50_kube-system_dca91be4d9d4850f3ca0a2251805edd7_0 -P /run/containers/storage/overlay-containers/0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c/userdata/conmon-pidfile -p /run/containers/storage/overlay-containers/0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c/userdata/pidfile --persist-dir /var/lib/containers/storage/overlay-containers/0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c/userdata -r /usr/local/bin/crun --runtime-arg --root=/run/runc --socket-dir-path /var/run/crio -u 0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c -s]" file="oci/runtime_oci.go:139"
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.641935163+08:00" level=debug msg="Running conmon under slice kubepods-burstable-poddca91be4d9d4850f3ca0a2251805edd7.slice and unitName crio-conmon-0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c.scope" file="cgmgr/systemd.go:140"
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.664709668+08:00" level=debug msg="Received container pid: -1" file="oci/runtime_oci.go:247" id=24eca757-1497-489c-8356-089cc43529ed name=/runtime.v1.RuntimeService/CreateContainer
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.664758979+08:00" level=error msg="Container creation error: creating cgroup directory `/sys/fs/cgroup/net_prio,net_cls/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca91be4d9d4850f3ca0a2251805edd7.slice/crio-0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c.scope`: No such file or directory\n" file="oci/runtime_oci.go:251" id=24eca757-1497-489c-8356-089cc43529ed name=/runtime.v1.RuntimeService/CreateContainer
Jan 28 16:34:26 vm10-0-135-50 crio[22057]: time="2023-01-28 16:34:26.670297621+08:00" level=info msg="createCtr: deleting container ID 0a858cdc7d77ea3bd5cc233c0732997701cd20dc1e437d72fc89e4005c1e7b6c from idIndex" file="server/container_create.go:497" id=24eca757-1497-489c-8356-089cc43529ed name=/runtime.v1.RuntimeService/CreateContainer

What did you expect to happen?

I wish it create conatiner sucessfully.

How can we reproduce it (as minimally and precisely as possible)?

1. install crio 1.23.4 from github
2. install kubelet 1.23.3
3. install and configue crictl
4. write a static yaml in /etc/kubernetes/manifests/
5. excute command 'crictl ps' check container

Anything else we need to know?

It work well on centos 8,with same crio version and k8s version.

CRI-O and Kubernetes version

```console $ crio --version crio version 1.23.4 Version: 1.23.4 GitCommit: a6a1e6ebf4c4baca7ca7d57f069a4e0cd89056cd GitTreeState: dirty BuildDate: 1980-01-01T00:00:00Z GoVersion: go1.17.4 Compiler: gc Platform: linux/amd64 Linkmode: static BuildTags: apparmor, exclude_graphdriver_devicemapper, seccomp, selinux SeccompEnabled: true AppArmorEnabled: false ``` ```console $ kubectl --version Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"} ```

OS version

```console # On Linux: $ cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/ $ uname -a Linux vm10-0-135-50 3.10.0-1160.59.1.el7.x86_64 #1 SMP Wed Feb 23 16:47:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux ```

Additional environment details (AWS, VirtualBox, physical, etc.)

[haircommander]

what version of crun? this looks like something there (or maybe a misconfiguration of the runtime spec) cc @giuseppe @flouthoc

[hanfengyizu]

what version of crun? this looks like something there (or maybe a misconfiguration of the runtime spec) cc @giuseppe @flouthoc

the version of crun as follow:

# crun --version
crun version 0.20.1
commit: 38271d1c8d9641a2cdc70acfa3dcb6996d124b3d
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL

[flouthoc]

This is a really old version of crun. I'd recommend upgrading it.

[giuseppe]

also any chance to use anything newer than RHEL 7? crun was barely tested there, and stuff like systemd cgroups won't probably work

[hanfengyizu]

also any chance to use anything newer than RHEL 7? crun was barely tested there, and stuff like systemd cgroups won't probably work

It work well on centos8.0 using the same cri-o package. Thank you for your answers

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[haircommander]

are we good to close this @hanfengyizu ?

[hanfengyizu]

are we good to close this Ok.