search

Helsinki-NLP / Opus-MT

Open neural machine translation models and web services
MIT License
574 stars 71 forks source link

Use more than one GPU to increase translations speed #75

Open edvardasast opened 1 year ago

edvardasast commented 1 year ago

Hi,

I would like to use more than one GPU to increase translations speed is it possible? I am using docker container with GPU support. As you could see in the screenshot bellow now it is using only one GPU at a time. https://imgur.com/a/OuBoAWg

martin-kirilov commented 1 year ago

Could you try adding --gpus all to your nvidia-docker run command? Something like: nvidia-docker run --gpus all -p 8888:8888 opus-mt-gpu:latest

edvardasast commented 1 year ago

Could you try adding --gpus all to your nvidia-docker run command? Something like: nvidia-docker run --gpus all -p 8888:8888 opus-mt-gpu:latest

Yes, I have done that, but it looks that system is not for multi gpu processing because it use only one GPU at a time and if I increase load translations speed dropping, but system do not start to use more GPUs for processing requests.