search

NVIDIA / nvidia-container-toolkit

Build and run containers leveraging NVIDIA GPUs
Apache License 2.0
2.45k stars 261 forks source link

Use cgroup v2 in rootless mode #189

Open moetayuko opened 2 years ago

moetayuko commented 2 years ago

NVIDIA Container Toolkit doesn't work in rootless mode by default, because cgroup is not supported in rootless mode, disabling its use fixed the issue as mentioned in https://github.com/NVIDIA/nvidia-docker/issues/1155#issuecomment-884499204

However, limiting resources is supported in rootless mode when running with cgroup v2 and systemd according to docker docs, is it possible to migrate to cgroup v2 so we don't need to set no-cgroups in rootless mode?

klueska commented 2 years ago

Support for cgroupv2 was added in libnvidia-container 1.8.0.

I've not followed those docs for rootless docker, but rootless podman works with cgroupv2 on the latest libnvidia-container.

moetayuko commented 2 years ago

Support for cgroupv2 was added in libnvidia-container 1.8.0.

Yes, it works out-of-box on Debian 11 (which uses cgroupv2) when docker runs as root user, but I have to set no-cgroups in /etc/nvidia-container-runtime/config.toml to make it work when docker is rootless. Otherwise it errors as https://github.com/NVIDIA/nvidia-docker/issues/1155#issue-536921053 on docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

bhack commented 2 years ago

Any news on this?