huggingface / accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
https://huggingface.co/docs/accelerate
Apache License 2.0
7.77k stars 941 forks source link

Added Multicpu SLURM example .sh script in examples/slurm folder #2824

Closed okhleif-IL closed 4 months ago

okhleif-IL commented 4 months ago

This PR contributes a multicpu SLURM example script in the examples/slurm folder.

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.