I needed some extra environment variables in order for KoboldAI to see my GPU (nvidia) inside the container. Figured this should be documented somewhere.
For reference, kubernetes manifest. I use metallb and some persistent storage, so the Deployment section may be the only part that anyone cares about. I'm running this on microk8s with the gpu plugin enabled.
I needed some extra environment variables in order for KoboldAI to see my GPU (nvidia) inside the container. Figured this should be documented somewhere.
Specifically, I needed these
NVIDIA_DRIVER_CAPABILITIES=compute,utility NVIDIA_VISIBLE_DEVICES=all
For reference, kubernetes manifest. I use metallb and some persistent storage, so the Deployment section may be the only part that anyone cares about. I'm running this on microk8s with the gpu plugin enabled.