Performance Settings

[KSimulation]

Thanks for putting together such a cool project!

I wonder what performance settings I can change to boost the translation speed with a higher-end GPU or even multiple GPUs. I was trying it on an RTX3090 and noticed that the GPU usage is always below 30-40%.

I have played around with some settings but didn't seem to do the trick. Since the pipeline is quite long and involves multiple models, could you point out some performance settings that can speed up the translation?

Thanks in advance.

[thatDudo]

There is not much, you can obviously choose the bigger translator models if you choose to use offline translators but otherwise im not sure what might cause the bottleneck.. Maybe the cpu or disk read write speed? Perhaps in the future a mode could be implemented wherein multiple manga images could be translated concurrently.

[KSimulation]

When running a batch job, the CPU(13900K) usage is often below 20% and the disk read/write speed is below 30MB/sec... I think we need to figure out a way to do more parallel processing at each stage.

[rspreet92]

@dmMaze has implemented a pretty good batch processing in BallonsTranslator

[zyddnys]

Running tasks in batches is possible but it will not fully utilize a high end GPU as the bottleneck is going to be the CPU.

[thatDudo]

Perhaps what is currently most impactful is single thread performance, which explains the 20% of cpu usage. So making the batch translation multi threaded might be well advised.

[ryanolee]

Translation is normally limited by the translation service you are using. If you are not using the offline modes everything gets sent over the network to an external API meaning it is entirely dependant on the external service to return before things can begin processing again. You will only see performance gains (At least for actual text translation) from having a larger GPU when specifically using the offline translation models :eyes:

[singersbalm]

You could increase the performance with jit or a compiler like cython