[BUG] ViT export stopped working correctly - Githubissues

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

https://aihub.qualcomm.com

BSD 3-Clause "New" or "Revised" License

417 stars 58 forks source link

[BUG] ViT export stopped working correctly #73

Open Lavreniuk opened 1 month ago

Lavreniuk commented 1 month ago

The model inference time is much slower

To Reproduce pip install -q qai-hub-models git clone https://github.com/quic/ai-hub-models cd ai-hub-models/qai_hub_models/models/vit python -m qai_hub_models.models.vit.export

Expected behavior here is expected results: https://aihub.qualcomm.com/jobs/jn5qv9l7g

Stack trace here is my run: https://app.aihub.qualcomm.com/jobs/jmg9drmw5/ (in your example all layers are in NPU, in my not).

Also if I download weights from your repo from this commit: https://huggingface.co/qualcomm/VIT/tree/973054da6d5d65b53537f7e608021bded3c4c522 it works fine, but the newest commit it same issue as when I converted by myself (slow inference time).

Host configuration:

OS and version: Ubuntu 22.04.3 LTS
QAI-Hub-Models version: AI Hub : aihub-2024.07.11.0 (in aihub-2024.06.27.0 I got good inference speed, but don't know how I could downgrade with version)
qai-hub-models==0.9.2 and 0.10.0 (tried both).

mestrona-3 commented 1 month ago

Hi @Lavreniuk , thank you for filing this issue! Apologies for the delayed response. This is a known issue due to one of package dependencies in AI Hub. We will fix this once resolved in the dependent package. Please join our Slack Community for release notes on when this issue is resolved.