Qualcomm® AI Hub Models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for deployment on Qualcomm® devices.

• List of Models by Category
• On-Device Performance Data
• Device-Native Sample Apps

See supported: On-Device Runtimes, Hardware Targets & Precision, Chipsets, Devices

Setup

1. Install Python Package

The package is available via pip:

# NOTE for Snapdragon X Elite users:
# Only AMDx64 (64-bit) Python in supported on Windows.
# Installation will fail when using Windows ARM64 Python.

pip install qai_hub_models

Some models (e.g. YOLOv7) require additional dependencies that can be installed as follows:

pip install "qai_hub_models[yolov7]"

2. Configure AI Hub Access

Many features of AI Hub Models (such as model compilation, on-device profiling, etc.) require access to Qualcomm® AI Hub:

• Create a Qualcomm® ID, and use it to login to Qualcomm® AI Hub.
• Configure your API token: qai-hub configure --api_token API_TOKEN

Getting Started

Export and Run A Model on a Physical Device

All models in our directory can be compiled and profiled on a hosted Qualcomm® device:

pip install "qai_hub_models[yolov7]"

python -m qai_hub_models.models.yolov7.export [--target-runtime ...] [--device ...] [--help]

Using Qualcomm® AI Hub, the export script will:

1. Compile the model for the chosen device and target runtime (see: Compiling Models on AI Hub).
2. If applicable, Quantize the model (see: Quantization on AI Hub)
3. Profile the compiled model on a real device in the cloud (see: Profiling Models on AI Hub).
4. Run inference with a sample input data on a real device in the cloud, and compare on-device model output with PyTorch output (see: Running Inference on AI Hub)
5. Download the compiled model to disk.

End-To-End Model Demos

Most models in our directory contain CLI demos that run the model end-to-end:

pip install "qai_hub_models[yolov7]"
# Predict and draw bounding boxes on the provided image
python -m qai_hub_models.models.yolov7.demo [--image ...] [--on-device] [--help]

End-to-end demos:

1. Preprocess human-readable input into model input
2. Run model inference
3. Postprocess model output to a human-readable format

Many end-to-end demos use AI Hub to run inference on a real cloud-hosted device (if the --on-device flag is set). All end-to-end demos also run locally via PyTorch.

Sample Applications

Native applications that can run our models (with pre- and post-processing) on physical devices are published in the AI Hub Apps repository.

Python applications are defined for all models (from qai_hub_models.models.\ import App). These apps wrap model inference with pre- and post-processing steps written using torch & numpy. These apps are optimized to be an easy-to-follow example, rather than to minimize prediction time.

Model Support Data

On-Device Runtimes

Device Hardware & Precision

*Some older chipsets do not support fp16 inference on their NPU.

Chipsets

• Snapdragon 8 Elite, 8 Gen 3, 8 Gen 2, and 8 Gen 1 Mobile Platforms
• Snapdragon X Elite Compute Platform
• SA8255P, SA8295P, SA8650P, and SA8775P Automotive Platforms
• QCS 6490, QCS 8250, and QCS 8550 IoT Platforms
• QCS8450 XR Platform

and many more.

Devices

• Samsung Galaxy S21, S22, S23, and S24 Series
• Xiaomi 12 and 13
• Snapdragon X Elite CRD (Compute Reference Device)
• Qualcomm RB3 Gen 2, RB5

and many more.

Model Directory

Computer Vision

Audio

Multimodal

Generative Ai

Need help?

Slack: https://aihub.qualcomm.com/community/slack

GitHub Issues: https://github.com/quic/ai-hub-models/issues

Email: ai-hub-support@qti.qualcomm.com.

LICENSE

Qualcomm® AI Hub Models is licensed under BSD-3. See the LICENSE file.