Open jordimas opened 1 year ago
Hello!
First, thanks for writing such a great tool.
Whisper.cpp: version 1.20 Open AI: version openai-whisper-20230124 Model used: medium
Audio file used: https://github.com/jordimas/whisper-cpp-error/raw/main/15GdH9-curt.mp3 Open AI transcription: https://raw.githubusercontent.com/jordimas/whisper-cpp-error/main/15GdH9-curt/15GdH9-curt.mp3.txt Whisper.cpp: transcription: https://raw.githubusercontent.com/jordimas/whisper-cpp-error/main/15GdH9-curt.wav.txt
I will expect Whisper.cpp to produce the same output under the same model and input than OpenAI Whisper.
In terms of WER against reference the txt human transcribed file: OpenAI whisper -WER: 28.08, Whisper.cpp : WER 35.86
If there is anything that I can do to help, let me know
Thanks
Thanks for the data point! How do I calculate WER scores?
Basically:
However, you can also see that the produced files are different.
Hello!
First, thanks for writing such a great tool.
Whisper.cpp: version 1.20 Open AI: version openai-whisper-20230124 Model used: medium
Audio file used: https://github.com/jordimas/whisper-cpp-error/raw/main/15GdH9-curt.mp3 Open AI transcription: https://raw.githubusercontent.com/jordimas/whisper-cpp-error/main/15GdH9-curt/15GdH9-curt.mp3.txt Whisper.cpp: transcription: https://raw.githubusercontent.com/jordimas/whisper-cpp-error/main/15GdH9-curt.wav.txt
I will expect Whisper.cpp to produce the same output under the same model and input than OpenAI Whisper.
In terms of WER against reference the txt human transcribed file: OpenAI whisper -WER: 28.08, Whisper.cpp : WER 35.86
If there is anything that I can do to help, let me know
Thanks