search

SYSTRAN / faster-whisper

Faster Whisper transcription with CTranslate2
MIT License
12.26k stars 1.03k forks source link

Question about compute evaluation metric #415

Open DuyTa506 opened 1 year ago

DuyTa506 commented 1 year ago

Hi there , i'm really excited to your repo , but i have a question , after quantized , how did we calculate the WER metric ? Will it different from whisper base model or just load the same architechture but in lower precision floating-point ?

guillaumekln commented 1 year ago

I'm not sure to understand the question. Are you asking how to compute the WER score?

DuyTa506 commented 1 year ago

i'm trying to calculate the wer metric , running time on this repo and huggingface model , but it's seem the running time on this repo take so much more than the other version, i dont know why . The model run on faster-whisper was quantized to int8_float16 type. Is that the reason?

guillaumekln commented 1 year ago

So is the issue the running time difference between faster-whisper and the Hugging Face model?

Please post the code you are using to run faster-whisper and the Hugging Face model.