search

NVIDIA / nvidia-container-toolkit

Build and run containers leveraging NVIDIA GPUs
Apache License 2.0
2.28k stars 245 forks source link

Question: How to assign container replicas to different GPUs #521

Open kheyer opened 4 months ago

kheyer commented 4 months ago

I have a container service in a docker-compose system that runs on a single GPU. Since I have multiple GPUs, I would like to create replicas by adding 

replicas: ${GPU_COUNT}

To the docker-compose config. The issue is doing this causes all replicas to use the same GPU. Currently I work around this by duplicating the service specification multiple times in the docker-compose config, manually assigning each replica to a different GPU.

Is there a good way to automatically assign replicas to different GPUs?