search

argosopentech / argos-translate

Open-source offline translation library written in Python
https://www.argosopentech.com
MIT License
3.7k stars 272 forks source link

Performance issues with GPU is it normal? #199

Closed matbht closed 2 years ago

matbht commented 2 years ago

Hi,

I have installed argos-translate, running it with GPU acceleration and it's taking 9 seconds to translate only those small strings..

1 => Insomnia is a very common sleep disorder in Western countries and would affect almost 20% of the adult population. 2 => Its frequency would be higher in women and increases with age (with age deep and paradoxical sleep decrease, the number and duration of night awakenings increase, 3 => Being unemployed, living alone or suffering from chronic somatic disease (cardiovascular disease, pain...) are additional risk factors for insomnia. 4 => According to some data, people with anxiety or depression would have 7 to 10 times more risk of chronic insomnia than others.

Time: 00:00:09

I'm running it on a $900 / month server with only this on it:

Model T2-45 Memory 45 Go Processor 15 vCores Stockage 400 Gio Debian 10 Cuda

Model

Tesla V100S 32 GB Cuda cores 5120

https://www.pny.eu/en/consumer/explore-all-products/nvidia-tesla-ai-cards/1279-tesla-v100s-32gb

Is there anything I'm doing wrong?

ARGOS_DEVICE_TYPE=cuda argos-translate --from-lang en --to-lang fr "Hello world"

Let me know thanks

EDIT: If I use argos-translate-gui it's so much faster because the software is launched and doesn't restart each time I want to translate.

In code I'm running for each string the command ARGOS_DEVICE_TYPE=cuda argos-translate --from-lang en --to-lang fr $string, so maybe it's restarting each time the program to launch it and translate, here is maybe the issue.

Do you know how I can let it run to be able to translate faster without restarting the API of translation of the CLI?

Thanks

PJ-Finlay commented 2 years ago

The nature of Python I think it parses Argos Translate and all of its dependencies on every CLI call. For example, I think even this is somewhat slow:

argos-translate --help

Making multiple calls from a single Python script translating directly or calling a remote LibreTranslate instance should avoid this issue.

matbht commented 2 years ago

Thank you :)

how can I activate GPU with libretranslate?

PJ-Finlay commented 2 years ago

LibreTranslate is using Argos Translate so setting the same environment variable should do it:

export ARGOS_DEVICE_TYPE=cuda
argos-translate --from-lang en --to-lang fr "Hello World"
PJ-Finlay commented 2 years ago

https://github.com/argosopentech/argos-translate#gpu-acceleration