[training] Updates for distil-large-v3

huggingface / distil-whisper

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

MIT License

3.32k stars 238 forks source link

Closed sanchit-gandhi closed 3 months ago

sanchit-gandhi commented 3 months ago

Updates the training code to reflect the changes implemented in distil-large-v3. These changes provide:

Drastically improved WER performance when using the sequential long-form decoding algorithm (as detailed here)
Higher throughput when training
Checkpoints that are compatible with the decoding algorithms used in other Whisper libraries

cc @bofenghuang @flozi00

bofenghuang commented 3 months ago

Fantastic!