softwarefactory-project-zuul[bot]
commented
1 month ago Build succeeded. https://softwarefactory-project.io/zuul/t/local/buildset/1d461ea631cd4b67ab19f6a111cc4d73
:heavy_check_mark: unit-test SUCCESS in 6m 52s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 54s :heavy_check_mark: unit-test-restricted SUCCESS in 6m 02s :heavy_check_mark: system-test-fedora-rawhide SUCCESS in 39m 11s :heavy_check_mark: system-test-fedora-40 SUCCESS in 35m 38s :heavy_check_mark: system-test-fedora-39 SUCCESS in 35m 21s :heavy_check_mark: system-test-fedora-38 SUCCESS in 34m 56s
debarshiray
This uses the NVIDIA Container Toolkit [1] to generate a Container Device Interface specification [2] on the host during
enterandruncommands. The specification is saved as JSON in the runtime directories at/run/toolboxor$XDG_RUNTIME_DIR/toolboxto make it available to the Toolbx container's entry point. The environment variables mentioned in the specification are directly passed topodman exec, while the hooks and mounts are handled by the entry point.Toolbx containers already have access to all the devices in the host operating system's
/dev, and containers share the kernel space driver with the host. So, this is only about making the user space driver available to the container. It's done by bind mounting the files mentioned in the generated CDI specification from the host to the container, and then updating the container's dynamic linker cache.This neither depends on
nvidia-ctk cdi generateto generate the Container Device Interface specification nor onpodman create --deviceto consume it.The main problem with
nvidia-ctkandpodman createis that the specification must be saved in/etc/cdior/var/run/cdi, both of which require root access, for it to be visible topodman create --device. Toolbx containers are often used rootless, so requiring root privileges for hardware support, something that's not necessary on the host, will be a problem.Secondly, updating the
toolbox(1)binary won't let existing containers use the proprietary NVIDIA driver, becausepodman createonly affects new containers.Therefore,
toolbox(1)uses the Go APIs used bynvidia-ctk cdi generateandpodman create --deviceto generate, save, load and apply the CDI specification itself. This removes the need for root privileges due to/etc/cdior/var/run/cdi, and makes driver avaiable to existing containers.Based on an idea from Ievgen Popovych.
[1] https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/ https://github.com/NVIDIA/nvidia-container-toolkit
[2] https://github.com/cncf-tags/container-device-interface
https://github.com/containers/toolbox/issues/116