tflite models - segmentation fault

[cogmeta]

I tried using tflite models built using the notebook https://github.com/usefulsensors/openai-whisper/blob/main/notebooks/generate_tflite_from_whisper.ipynb.

But, I am getting segmentation fault when tried with stream. Any ideas why that might be happening?

[cogmeta]

n_vocab:50257 audio_sdl_init: found 1 capture devices: audio_sdl_init: - Capture device #0: 'MacBook Pro Microphone' audio_sdl_init: attempt to open capture device 0 : 'MacBook Pro Microphone' ... audio_sdl_init: obtained spec for input device (SDL Id = 2): audio_sdl_init: - sample rate: 16000 audio_sdl_init: - format: 33056 (required: 33056) audio_sdl_init: - channels: 1 (required: 1) audio_sdl_init: - samples per frame: 1024 INFO: Created TensorFlow Lite XNNPACK delegate for CPU. Segmentation fault: 11

[nyadla-sys]

Could you please follow the steps outlined here https://github.com/usefulsensors/openai-whisper/tree/main/stream

Sorry, I don't have macbook pro to verify the same

[cogmeta]

Oh, it works perfectly well with the ../models/whisper.tflite but not with the models created with the notebook. I wanted to try creating tflite with medium model.

[nyadla-sys]

Use the following colab, but swap out openai/whisper-tiny for openai/whisper-medium. https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/notebooks/generate_tflite_from_whisper.ipynb

[cogmeta]

Followed the exact same steps. Trying to use the model results in segmentation faults even on Ubuntu machine. Does not look like mac issue.

[nyadla-sys]

I may need to construct a new multilingual vocab bin, but it may be worthwhile to test the model below for the existing vocab bin Use the following GitHub repository, but replace openai/whisper-tiny with openai/whisper-medium.en.

[nyadla-sys]

Before building stream example replace ~/openai-whisper/stream/filters_vocab_gen.h with filters_vocab_multilingual.h cp ~/openai-whispermodels/filters_vocab_multilingual.h ~/openai-whispermodels/stream/filters_vocab_gen.h then follow the build steps and run with below command to use whisper-medium.tflite model on stream example ./stream ../models/whisper-medium.tflite

[nyadla-sys]

Use whisper-medium.tflite that you generated

[nyadla-sys]

I am not able to upload model as it is around 700MB in size

[nyadla-sys]

I could add whisper-medium.tflite

[cogmeta]

Thanks! will try it out.

[cogmeta]

./stream ../models/whisper-medium.tflite

n_vocab:50257 audio_sdl_init: found 1 capture devices: audio_sdl_init: - Capture device #0: 'MacBook Pro Microphone' audio_sdl_init: attempt to open capture device 0 : 'MacBook Pro Microphone' ... audio_sdl_init: obtained spec for input device (SDL Id = 2): audio_sdl_init: - sample rate: 16000 audio_sdl_init: - format: 33056 (required: 33056) audio_sdl_init: - channels: 1 (required: 1) audio_sdl_init: - samples per frame: 1024 INFO: Created TensorFlow Lite XNNPACK delegate for CPU. ERROR: gather index out of bounds ERROR: Node number 35 (GATHER) failed to invoke. ERROR: Node number 8319 (WHILE) failed to invoke. Error at /Users/prashantsasatte/openai-whisper/tensorflow_src/tensorflow/lite/examples/stream/stream.cc:366

[nyadla-sys]

Can u try with minimal build and replace filters vocab gen.bin with filters vocab multilingual and rename as filters vocab gen.bin

[nyadla-sys]

I think it is producing more tokens than I restricted to 223 in model generation,pls change max_tokens to 384 and generate model again

[cogmeta]

Tried with minimal build. same error...I have not yet looked into the code. will do that. Thanks.

I am actually looking to implement a streaming GRPC server.

[nyadla-sys]

I have incorporated 448 tokens into the whisper-medium model in an attempt to solve your issue.

[nyadla-sys]

please try latest whisper-medium model