Decide on Whisper implementation

[Veldhoen]

Overview of computational performance (time & memory - also for longform) ASR performance (WER) Project quality (well maintained software, no weird depencencies etc)

Presumably Faster Whisper or WhisperX

[greenw0lf]

Proposal: Use the PR from faster-whisper that introduces batching + add the piece of code from WhisperX that does diarization

Reasons why:

• faster-whisper with batching is faster than WhisperX w/o speaker diarization
• For WhisperX, numbers and currencies must be written out ('thirteen' instead of 13, 'euro' instead of €, etc.) because a separate alignment model based on wav2vec2 is used, which was trained to match audio units to letters only.
• faster-whisper uses Whisper's attention scores directly and can output timestamps for numbers and currencies, unlike WhisperX
• In terms of WER, faster-whisper scores a lower WER than WhisperX

[greenw0lf]

I wrote a document that details the decision: https://docs.google.com/document/d/17rVrum6L8qlsewoR36ricKwTtQMVWrayg8xfDkmPMRY/edit?usp=sharing