search

usefulsensors / openai-whisper

Robust Speech Recognition via Large-Scale Weak Supervision
MIT License
62 stars 24 forks source link

Support for --initial_prompt feature? #18

Open ankushg opened 1 year ago

ankushg commented 1 year ago

Thanks for this awesome resource!

The upstream whisper project added an --initial_prompt https://github.com/openai/whisper/pull/128

Is there a way to use that with this model?

steveturner commented 1 year ago

Interested in this as well -- it looks like the TFWhisperForConditionalGeneration might need to be used.

I tried a few things but didn't see an obvious way to preload a tensor with the vocabulary/prompt and then parse the logits out when using Tensorflow. But I'm also a noob.

@nyadla-sys has some good discussion around the topic here https://github.com/huggingface/transformers/issues/19691

nyadla-sys commented 1 year ago

@ankushg I have not looked into adding --initial_prompt for tflite whisper model as I am stuck in debugging full int8 model for whisper. Hopefully i will give sometime next week on this