search

quic / ai-hub-apps

The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
BSD 3-Clause "New" or "Revised" License
65 stars 14 forks source link

[question] I hope to know what app.aihub.qualcomm do to our models? #14

Closed ecccccsgo closed 1 day ago

ecccccsgo commented 1 week ago

hello, what you are doing is an amazing thing. and i'm working to test your toolkits to deploy models on Android devices. I counter some problems:

  1. the network problem..... due to large size of LLM, it took a long time to unload my model to your server, and downloaded it, if no accident, accident comes. the transportation broken for serval hours. I hope to know if i can do it on my local host? to save time and my mood....
  2. I hope to developed my model on SA8255 or 8 Gen1, and it's not on the support list. i think it may works.... because these chipsets have not bad NPU. and see these on devices = hub.get_devices().
  3. I hope to know why some version qairt have hexagon-v68 hexagon-v69 libraries like qairt/.26.0.240828 and some version not ( such as qairt/2.27.7.241014 only have hexagon-v75 and hexagon-v79)

looking forward to your reply. good days

mestrona-3 commented 1 week ago

Hi @ecccccsgo! Thank you for sharing your questions. I'll try to answer each of them: 1) We've made several improvements to the upload time and networking issues that we've experienced, and released these with qai-hub 0.19.0. Can you please try updating and let us know if you still experience these issues? If so, we'd love more information so we can continue to improve this experience. 2) The NPU in the 8gen1 is an older version than the following generations of mobile SoCs (8gen2, gen3 8 elite etc.) It may or may not support your model, we encourage you to try targeting the device with your desired model and post any failed jobs on slack, however if it is an LLM that we have not specific works on that device, then you'll need to target a newer device. 3) this is a question for the QNN team, we can pass it along to them but aren't sure on this ourselves.