Mass_unsup has no problem on a single GPU, and errors are reported on multiple GPUs

microsoft / MASS

MASS: Masked Sequence to Sequence Pre-training for Language Generation

https://arxiv.org/pdf/1905.02450.pdf

Other

1.11k stars 206 forks source link

Mass_unsup has no problem on a single GPU, and errors are reported on multiple GPUs #175

Closed MayDomine closed 2 years ago

MayDomine commented 2 years ago

I try to pretrain a unsup_MASS on my monolingual corpus for zn-en task, and everything is ok when I use a sigle gpu.But I met this when I try to use two gpus. here is the problem and my scripts.