Open MayuraRam opened 2 months ago
hi, attention_mask
was removed from the whisper beam search inputs in ort 1.16.0 so the inference example is outdated. Can you try after removing it from https://github.com/microsoft/onnxruntime-inference-examples/blob/0de2e66e03981714e5308c457b72d785e98d0fe2/js/ort-whisper/main.js#L144
Please refer here for more details of the model inputs https://github.com/microsoft/Olive/blob/main/examples/whisper/code/whisper_dataset.py#L50
Describe the bug Unable to optimize a model with device- cpu and precision int8. Ending up with KeyError: 'input_model' error
To Reproduce Start with this example: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/js/ort-whisper
Readme says:
Goto: https://github.com/microsoft/Olive/tree/main/examples/whisper and follow the instructions.
Run the following commands
Move the resulting model from models/whisper_cpu_int8_0_model.onnx to the same directory as this code.
When I did the above with a pip install of olive-ai, I go the KeyError: 'config' error.
Then I tried installing from source as mentioned here - https://github.com/microsoft/Olive/blob/main/examples/README.md
git clone https://github.com/microsoft/Olive.git cd Olive python -m pip install .
Then I tried to "Run the config to optimize the model" from here - https://github.com/microsoft/Olive/blob/main/examples/whisper/README.md
This script runs and creates\Olive-main\examples\whisper\models\conversion-transformers_optimization-onnx_dynamic_quantization-insert_beam_search-prepost\whisper_cpu_int8_cpu-cpu_model.onnx
(olive_env)\Olive-main\examples\whisper>python test_transcription.py --config \Olive-main\examples\whisper\models\conversion-transformers_optimization-onnx_dynamic_quantization-insert_beam_search-prepost\whisper_cpu_int8_cpu-cpu_model.json
Traceback (most recent call last):
File "\Olive-main\examples\whisper\test_transcription.py", line 126, in
output_text = main()
^^^^^^
File "\Olive-main\examples\whisper\test_transcription.py", line 63, in main
model_name = config["input_model"]["model_components"][0]["model_path"]