[training] enable ddp during distillation

huggingface / distil-whisper

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

MIT License

3.52k stars 280 forks source link

Closed sanchit-gandhi closed 5 months ago

sanchit-gandhi commented 5 months ago

Small bug fixes to enable ddp during distillation with accelerate. cc @eustlb - you can pull these changes before running any ddp training runs.