XLMR-xxl in multi-GPU - Githubissues

facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

MIT License

30.43k stars 6.4k forks source link

XLMR-xxl in multi-GPU #4102

Closed ahtamjan closed 2 years ago

ahtamjan commented 2 years ago

XLM-R xxl is too large to load in one GPU, how can I fix this? I have 4 V100-32G GPUs. Thanks

stale[bot] commented 2 years ago

This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

stale[bot] commented 2 years ago

Closing this issue after a prolonged period of inactivity. If this issue is still present in the latest release, please create a new issue with up-to-date information. Thank you!