search

nyadla-sys / whisper.tflite

Optimized OpenAI's Whisper TFLite Port for Efficient Offline Inference on Edge Devices
MIT License
134 stars 29 forks source link

generate_tflite_from_whisper.ipynb does no longer generate valid whisper-tiny.en.tflite model #34

Open ytsnko opened 3 months ago

ytsnko commented 3 months ago

https://github.com/nyadla-sys/whisper.tflite/blob/main/models/generate_tflite_from_whisper.ipynb

TFLite model is created without errors, but on the last cell where tflite is validated signature runner throws an error:

RuntimeError Traceback (most recent call last)

in <cell line: 6>() 4 5 tflite_generate = interpreter.get_signature_runner() ----> 6 generated_ids = tflite_generate(input_features=input_features)["sequences"] 7 transcription = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] 8 transcription

/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/interpreter.py in call(self, **kwargs) 247 self._subgraph_index) 248 --> 249 self._interpreter_wrapper.Invoke(self._subgraph_index) 250 result = {} 251 for output_name, output_index in self._outputs:

RuntimeError: tensorflow/lite/kernels/reduce.cc:445 std::apply(optimized_ops::Mean<T, U>, args) was not true.tensorflow/lite/kernels/reduce.cc:445 std::apply(optimized_ops::Mean<T, U>, args) was not true.tensorflow/lite/kernels/reduce.cc:445 std::appl...

I've tried to reduce max_new_tokens to 223 and that fixes the error, but the resulting model is recognizing garbage now:

this this, this this, this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this, this this this this this this this this this this this this this this this this this, this this this this this this this this this this, this this this this this this this this, this this this this this this this, this this this this this this, this this, this this, this this this this this, this this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this, this, this, this this, this, this, this, this,