Discussion on Multi-GPU Training

google / deepvariant

DeepVariant is an analysis pipeline that uses a deep neural network to call genetic variants from next-generation DNA sequencing data.

BSD 3-Clause "New" or "Revised" License

3.24k stars 729 forks source link

Discussion on Multi-GPU Training #885

Closed 0809zheng closed 2 months ago

0809zheng commented 2 months ago

Hi,

Since DeepVariant does not support multi-GPU training (Can model_train be run on multiple GPUs?), I am pretty curious about "We have tested training with 1 and 2 GPUs and observed the following runtimes:" mentioned in the training case. Specifically, how is the training with 2 GPUs tested?

Thank you! Regards : )

lucasbrambrink commented 2 months ago

Hi,

It is actually possible to train DeepVariant on multiple GPUs, using the MirroredStrategy. You can find the tensorflow documentation here: link.

It looks like we need to update the FAQ to reflect that—thanks for brining it to our attention! The training case study is up-to-date, so feel to continue to reference that. It looks like it already applies the mirrored strategy.