search

CurationCorp / curation-corpus

Code for obtaining the Curation Corpus abstractive text summarisation dataset
Creative Commons Attribution 4.0 International
123 stars 27 forks source link

Multi-GPU Finetuning of BART #6

Closed dakshvar22 closed 3 years ago

dakshvar22 commented 4 years ago

Hi,

Thanks for providing this notebook on finetuning BART. I want to use multiple GPUs for training since my dataset is quite huge. I see this line here which tries to set only one gpu to use: 

device=torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

How can I change the code to leverage multiple GPUs?

HenryDashwood commented 4 years ago

Sorry, about the slow reply!

It's not really something I explore in the tutorial. I am going to update it fairly soon though so perhaps I should add a section showing that.

The first place I would look would be here https://github.com/fastai/fastai2/blob/master/nbs/20a_distributed.ipynb