bug: significantly lower performance compared to original whisper.cpp

Describe the bug

I installed the latest version via:

pip install git+https://github.com/aarnphm/whispercpp.git -vv

Then I have this 1 minute long wav file.

Here's the output of the original whisper.cpp command:

``` ./main -m models/ggml-small.bin -f out.wav --language auto --max-len 1 whisper_init_from_file_no_state: loading model from 'models/ggml-small.bin' whisper_model_load: loading model whisper_model_load: n_vocab = 51865 whisper_model_load: n_audio_ctx = 1500 whisper_model_load: n_audio_state = 768 whisper_model_load: n_audio_head = 12 whisper_model_load: n_audio_layer = 12 whisper_model_load: n_text_ctx = 448 whisper_model_load: n_text_state = 768 whisper_model_load: n_text_head = 12 whisper_model_load: n_text_layer = 12 whisper_model_load: n_mels = 80 whisper_model_load: ftype = 1 whisper_model_load: qntvr = 0 whisper_model_load: type = 3 whisper_model_load: mem required = 743.00 MB (+ 16.00 MB per decoder) whisper_model_load: adding 1608 extra tokens whisper_model_load: model ctx = 464.68 MB whisper_model_load: model size = 464.44 MB whisper_init_state: kv self size = 15.75 MB whisper_init_state: kv cross size = 52.73 MB system_info: n_threads = 4 / 12 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 | COREML = 0 | main: processing 'out.wav' (958952 samples, 59.9 sec), 4 threads, 1 processors, lang = auto, task = transcribe, timestamps = 1 ... whisper_full_with_state: auto-detected language: ru (p = 0.993210) # recognition results go here ... whisper_print_timings: load time = 571.00 ms whisper_print_timings: fallbacks = 0 p / 0 h whisper_print_timings: mel time = 310.98 ms whisper_print_timings: sample time = 378.25 ms / 426 runs ( 0.89 ms per run) whisper_print_timings: encode time = 11926.32 ms / 4 runs ( 2981.58 ms per run) whisper_print_timings: decode time = 9821.29 ms / 425 runs ( 23.11 ms per run) whisper_print_timings: total time = 23272.83 ms ```

Total execution time is 23 seconds.

Here's my python code which uses this library:

import time
from whispercpp import Whisper

start = time.time()

w = Whisper.from_pretrained(model_name="/whispercpp/models/ggml-small.bin")

w.params.with_language("auto")

print(w.transcribe_from_file("out.wav"))

end = time.time()
print(end - start)

And here's the output on the same file:

``` whisper_init_from_file_no_state: loading model from '/whispercpp/models/ggml-small.bin' whisper_model_load: loading model whisper_model_load: n_vocab = 51865 whisper_model_load: n_audio_ctx = 1500 whisper_model_load: n_audio_state = 768 whisper_model_load: n_audio_head = 12 whisper_model_load: n_audio_layer = 12 whisper_model_load: n_text_ctx = 448 whisper_model_load: n_text_state = 768 whisper_model_load: n_text_head = 12 whisper_model_load: n_text_layer = 12 whisper_model_load: n_mels = 80 whisper_model_load: f16 = 1 whisper_model_load: type = 3 whisper_model_load: mem required = 608.00 MB (+ 16.00 MB per decoder) whisper_model_load: adding 1608 extra tokens whisper_model_load: model ctx = 464.56 MB whisper_model_load: model size = 464.44 MB whisper_init_state: kv self size = 15.75 MB whisper_init_state: kv cross size = 52.73 MB whisper_full_with_state: auto-detected language: ru (p = 0.993206) # recognition results go here... 183.6768798828125 ```

Total execution time is 183s.

Difference is almost 9x times.

To reproduce

No response

Expected behavior

I'd expect the performance to be the same as can be seen in original whisper.cpp

Environment

Macbook pro 16, 2,6 GHz 6-Core Intel Core i7; 32 GB RAM Python 3.10 Latest versions of whisper.cpp and this library as of 5th June 2023.

aarnphm / whispercpp