Restrict dorado GPU memory usage ?

[RxLoutre]

Hi dorado team !

It is me again, sorry ;)

Dorado 0.5.0 was very polite and was only using a little bit less than half of the GPU memory (We have an Nvidia A100 with 80 Gb memory).

Dorado 0.6.0 takes it all, and I don't see any option to restrict this behavior.

I would like to be able to start 2 basecalling thread side by side on the same GPU at the same time, but with dorado 0.6.0 it does not seem possible.

Would it be possible to add an option to restrict GPU memory usage ?

Best,

Roxane

[HalfPhoton]

You can partially control the amount of GPU memory consumed by setting the --batchsize.

The automatic batch size algorithm tries to maximise the basecalling throughput on the GPU which appears as increases memory usage.

Kind regards, Rich

[tijyojwad]

Hi @RxLoutre - you can limit the amount of GPU memory used by dorado by manually setting the batch size using -b parameter. The batch size determined by dorado automatically is in the dorado logs at the beginning (so you can use half of that to reduce the memory by approx half).

We generally recommend running only one dorado process on a GPU at a time since memory usage can fluctuate a bit, and this can lead to OOM if multiple processes are using the same GPU. You can mitigate this by lowering the batch size further (but this will affect performance).

[RxLoutre]

Okay thank you for your answer ! I will considerate the batch size option when doing my calls. Thank you !