search

Const-me / Whisper

High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
Mozilla Public License 2.0
8.2k stars 702 forks source link

Let you know if you don't, there is a `faster whisper` than `whisper.cpp` #143

Open HaujetZhao opened 1 year ago

HaujetZhao commented 1 year ago

Another repo faster-whisper that uses CTranslate2 to inference is n times faster than whisper.cpp. And it supports word-level timestamp.

Just let you know this in case you don't.

And, is it possible to make faster-whisper use DirectX to accelerate more?

emcodem commented 1 year ago

Yeah i have seen this project, it would be interesting if anyone could invest some serious effort to compare faster-whisper and const-me version in terms of speed and quality. Also i wonder if these kind of projects that try to be backward compatible to original whisper commandline usage already have a mode for "live" transcription as well?

HaujetZhao commented 1 year ago

https://github.com/guillaumekln/faster-whisper/issues/151 Mentioned the realtime transcription, seems work using vad.

ghost commented 1 year ago

I'm a tech noob so take my comment with a grain of salt. I have used both and currently Const-me's version produces better results.

emcodem commented 1 year ago

@softlypink faster-whisper with Ctranslate2 has lots of magic builtin regarding the sampling strategy, but Const only has simple greedy search (which i think is totally fine for use with whisper model). For better comparison, try to disable all related logic e.g. using beam_size=1,best_of=1, temperature=0. Penalty and log_prop also seem to play a role but i didnt quickly find out how to disable them.

This way output should be more compareable but their strategy to salvage repeated output (using the temperature parameter) is also disabled so you'll get more loops again. It's a pity.

Murugurugan commented 1 year ago

This should get the highest priority, after all having faster-whisper added speed + support on AMD would be a huge win for everyone.