I have a challenging one for you. I have this working pretty easily on a bare metal machine, but gave up trying on proxmox.
Hardware:
Intel Xeon w7 chip
Nvidia rtx 4090 (not the primary GPU)
Intel or Nvidia Enterprise NICs cards
Software
Win 2025 Datacenter
Nvidia Cuda drivers
Windows containers (with docker ce install to manage win vms) (can't share Cuda but can share directml)
Hyper v, which can paravirtualize the 4090, and share the NICs via sr-iov/NUMA. (Paravirualize means multiple guests shared GPU via "Add-VMGpuPartitionAdapter", instead of pass-through via "Add-VMAssignableDevice")
Ubuntu WSL2, with a 2nd install of docker (for Linux machines) and Nvidia container toolkit to share gpu with wsl and it's docker containers.
The win host, wsl ubuntu, any wsl docker containers, and hyper guests all recognizing the Nvidia 4090 when "nvidia-smi.exe" is run. And Windows containers via docker recognizing the 4090 (but directx acceleration instead of Cuda since Nvidia container toolkit doesn't work with win containers).
Its the ultimate AI setup, and it's pretty easy to do baremetal, but haven't had much luck inside proxmox.
I have a challenging one for you. I have this working pretty easily on a bare metal machine, but gave up trying on proxmox.
Hardware: Intel Xeon w7 chip Nvidia rtx 4090 (not the primary GPU) Intel or Nvidia Enterprise NICs cards
Software
Win 2025 Datacenter
Nvidia Cuda drivers
Windows containers (with docker ce install to manage win vms) (can't share Cuda but can share directml)
Hyper v, which can paravirtualize the 4090, and share the NICs via sr-iov/NUMA. (Paravirualize means multiple guests shared GPU via "Add-VMGpuPartitionAdapter", instead of pass-through via "Add-VMAssignableDevice")
Ubuntu WSL2, with a 2nd install of docker (for Linux machines) and Nvidia container toolkit to share gpu with wsl and it's docker containers.
The win host, wsl ubuntu, any wsl docker containers, and hyper guests all recognizing the Nvidia 4090 when "nvidia-smi.exe" is run. And Windows containers via docker recognizing the 4090 (but directx acceleration instead of Cuda since Nvidia container toolkit doesn't work with win containers).
Its the ultimate AI setup, and it's pretty easy to do baremetal, but haven't had much luck inside proxmox.