Support for LLamma3.2 on Qualcomm A100

quic / cloud-ai-sdk

Qualcomm Cloud AI SDK (Platform and Apps) enable high performance deep learning inference on Qualcomm Cloud AI platforms delivering high throughput and low latency across Computer Vision, Object Detection, Natural Language Processing and Generative AI models.

https://quic.github.io/cloud-ai-sdk-pages/latest/

Other

55 stars 7 forks source link

Support for LLamma3.2 on Qualcomm A100 #12

Closed alew3 closed 1 month ago

alew3 commented 1 month ago

Do you plan on supporting Llama3.2 (text/multimodal) on Qualcomm A100? I saw this post (https://www.qualcomm.com/news/onq/2024/09/qualcomm-partners-with-meta-to-support-llama-3-point-2-big-deal-for-on-device-ai) , but it seems to have compatibility only for Snapdragon chips.

groundsada commented 5 days ago

Hi @alew3 I am curious to know whether you were able to load up llama3.2 on the Cloud AI 100