search

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
https://aihub.qualcomm.com
BSD 3-Clause "New" or "Revised" License
497 stars 78 forks source link

Running models on Windows on Snapdragon #12

Closed kristoftunner closed 8 months ago

kristoftunner commented 8 months ago

I would like to run the controlnet(https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/controlnet_quantized/README.md) models on the HTP on snapdragon with windows. Is it possible to run the exported binary cache files of the controlnet models on snapdragon with windows?

In the docs I saw that the only support runtime is qnn_lib_aarch64_android for job compilation.

bhushan23 commented 8 months ago

Hi @kristoftunner you can run exported context binary files on snapdragon with windows. Please make sure you are also using QNN SDK 2.19

We are in the actively validating these models on snapdragon on windows, it would be great if you can give it a spin and let us know how it goes :)

kristoftunner commented 8 months ago

@bhushan23 thx for the quick feedback, I will update you here once I have it running.

kory commented 8 months ago

Hi @kristoftunner , a clarification. You will need to use --target-runtime qnn_bin in your compile options to get a cross platform asset, if you are using hub. qnn_lib_aarch64_android will not work on Windows.

The binaries provided for this model (.bin) will work regardless :)

kristoftunner commented 8 months ago

@bhushan23 @kory the model binaries are working under windows, I tried it out on a volterra machine, thx