Runtime configuration of CDI spec dirs like CRI-O's --cdi-spec-dirs

[bartoldeman]

Feature request description

I couldn't find any way (without going into podman's source code) of changing the default from /etc/cri and /var/run/cri.

Use case: I'd like to dynamically generate a yaml file as normal user using e.g. nvidia-ctk cdi generate --output=/var/tmp/cdi/nvidia.yaml and make podman use it.

Suggest potential solution

An option --cdi-spec-dirs much like crio.

Have you considered any alternatives?

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@umohnani8 WDYT?

[umohnani8]

Hi @bartoldeman, are you talking specifically about the podman kube play use case? You want podman kube play to be able to look at a directory for the cdi specs and take that into account when playing the kube yaml or do you want podman kube generate to be able to look at a directory for the cdi specs and add it to the generated yaml?

Could you please give a bit more details on what exactly you expect with some examples? This will help us figure out best how to support the cdi spec dirs.

[nmcglo]

Seconding the initial reporter, I have the same wish if I'm reading his issue correctly.

It would be nice to be able to point podman to a user defined location for a CDI specification .yaml. Currently it expects those specifications to be placed in typically root owned directories.

A CDI specification can be generated from nvidia container toolkit without sudo privileges but placing it in the locations where podman is ~hardcoded~ defaulted to look for them does require sudo privileges. So even if the nvidia container toolkit was already installed on the system by an admin and made accessible for users, actually using CDI specifications -- if it wasn't already set up and maintained by an admin -- still requires sysadmin intervention.

CDI can replace the --hooks-dir method of configuring devices for mapping into containers. --hooks-dir could be specified to point to a user-owned directory when using podman run, could we have the same thing for CDI such as --cdi-dir?

[bartoldeman]

yes, sorry for the delay, and thanks for the reminder this has nothing to do with kube The workflow would be (as a non-root user!)

nvidia-ctk cdi generate --output=/tmp/$USER/cdi/nvidia.yaml
podman run --cdi-spec-dirs=/tmp/$USER/cdi --ipc host --rm --device nvidia.com/gpu=all ubuntu nvidia-smi -L

as a normal user can't write to either /etc/cdi or /var/run/cdi.

[nmcglo]

Possible pathway for modifying podman source to accommodate this:

Looks like something could be done by passing the necessary CLA option value to the config that eventually gets to the point where the CDI registry is fetched. Then the option could be utilized with reg.Configure(cdi.WithSpecDirs(<PARSED CLA OPTION VALUE>...)) similar to what containerd did.

[micahcc]

Also running into this when I try to make podman containers rootless and reusable as part of a nix package. Would definitely be nice to be able to pass in cdi spec paths as an argument. For now I'm probably going to do something gross like copy container.conf to some tempfile, write this in, then wrap the podman binary with the updated CONTAINER_CONF=/tmp/container.conf. Kind of ugly. Honestly would be amazing if every key in container configuration was overridable by command line.

[micahcc]

Went ahead and submitted a PR, because why not