Finetune with multiple GPU

google-research / albert

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Apache License 2.0

3.23k stars 571 forks source link

Finetune with multiple GPU #228

Open ReyonRen opened 3 years ago

ReyonRen commented 3 years ago

Hi all, thanks for the cool contribution.

I noticed that TPUEstimator is used in the code. After running on a machine with multiple GPUs, I found that only one GPU is in use. How can I modify the code so that the code can run in parallel on multiple GPUs?

Thank you!