search

vilassn / whisper_android

Offline Speech Recognition with OpenAI Whisper and TensorFlow Lite for Android
MIT License
189 stars 27 forks source link

How to generate whisper tflite model #7

Open timlliu opened 6 months ago

timlliu commented 6 months ago

I want to generate my own model. I try : https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/notebooks/whisper_base_tflite_model.ipynb#scrollTo=TzCrY9Q5jVsg But, it's not work for me.

vilassn commented 6 months ago

https://github.com/nyadla-sys/whisper.tflite/tree/main/models

Use script from this link to generate model.

timlliu commented 6 months ago

I use multilingual model. image

Change forced_decoder_ids to [[1, 50260], [2, 50359], [3, 50363]]. image

Output is correct. image

tflite model's output is not correct image

Am I missing something?

scute0001 commented 5 months ago

i have the same problem.

https://github.com/nyadla-sys/whisper.tflite/tree/main/models i follow readme colab to generate multilingual model and replace in android sample but model is not work. and has below error msg. image

how can i get the multilingual model?

yong10202 commented 2 months ago

@timlliu If you proceed with 'monkey patch' referring to this comment, you can solve the problem that forced_decoder_ids does not apply to tflite.

https://github.com/nyadla-sys/whisper.tflite/discussions/15#discussioncomment-7362798

When converting to tflite, it seems that the specific code of the hugging face caused the problem.