vade / OpenAI-Whisper-CoreML

OpenAI's Whisper ported to CoreML
144 stars 3 forks source link

Update model to use Key Value Caching similar to Pytorch #7

Open vade opened 1 year ago

vade commented 1 year ago

See this comment from Matthijs the CoreML ninja :D

https://twitter.com/mhollemans/status/1618639882402025473?s=46&t=bOTr_fvEuCVEiBiGAGuSKw

Im guessing steps are to:

vade commented 1 year ago

see : https://scale.com/blog/pytorch-improvements#Text%20Translation

and

https://github.com/openai/whisper/blob/f82bc59f5ea234d4b97fb2860842ed38519f7e65/whisper/decoding.py#L134