search

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
https://aihub.qualcomm.com
BSD 3-Clause "New" or "Revised" License
338 stars 45 forks source link

Exceptions while trying to run Whisper ASR example #19

Closed cfasana closed 3 months ago

cfasana commented 3 months ago

I was trying to run the Whisper ASR example. However, I encountered some issues which I would like to point out.

  1. After running the following lines: 
    pip install "qai_hub_models[whisper_asr]"
python -m qai_hub_models.models.whisper_asr.demo

    an exception is raised as reported in https://github.com/quic/ai-hub-models/issues/15.

  2. Solving that as reported in the same issue and proceeding with the export commands python -m qai_hub_models.models.whisper_asr.export raises another exception which is equivalent to the one before and that can be fixed in a similar manner inside the source code.
  3. Then, trying to execute the command again gives another exception during the profiling step: 
    qai_hub/client.py", line 1097, in _determine_model_type
raise UserError(qai_hub.client.UserError: Unsupported model type. The following types are supported
    - TorchScript: Extension .pt or .pth
    - Core ML Model: Extension .mlmodel
    - Compiled Core ML Model: Extension .mlmodelc or .mlmodelc.zip
    - Tensorflow Lite: Extension .tflite
    - ONNX: Extension .onnx
    - ONNX (ORT model format): Extension .ort
    - QNN Binary: Extension .bin
    - AIMET Model: Directory ending with .aimet
  4. Trying to skip the profiling operation (python -m qai_hub_models.models.whisper_asr.export --skip-profiling) can of course avoid that the previous exception is encountered, but gives another exception similar to the first one: 
    qai_hub_models/utils/base_model.py", line 187, in sample_inputs
inputs_dict[input_name] = [inputs_list[i].numpy()]
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

    Why is it not possible to proceed with the profiling operation? The same issue is also encountered when trying to run the inference step, but I guess that these two problems are due to the same reason. I hope that the information I reported can be helpful.

shreyajn commented 3 months ago

Thanks for reporting the issue. We are actively working on a fix so the next release should have it.

cfasana commented 3 months ago

@shreyajn, I tested the new release v0.4.0r1 of the AI Hub and did not experience these issues anymore. I will close this issue and open it again only if I encounter similar problems. Thanks for the support!