Do u have plan to building and running Llama2 model with Qualcomm AI Engine Direct Backend?

pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch

https://pytorch.org/executorch/

Other

2.1k stars 343 forks source link

Do u have plan to building and running Llama2 model with Qualcomm AI Engine Direct Backend? #1799

Open kyungsoo3lee opened 9 months ago

kyungsoo3lee commented 9 months ago

Hi. I am applying Llama2 model with Qualcomm Chip(SM8650, SM8550). I can't find qualcomm scripts of llama2 model for Qualcomm AI Engine Direct and running a model on it.

Could u give me a any information?

guangy10 commented 9 months ago

@larryliu0820 Is the llama2 demo supported on qualcomm script?

cccclai commented 9 months ago

We're still working on it. Here is the doc regarding the models supported with Qualcomm backend currently

https://github.com/pytorch/executorch/tree/main/examples/qualcomm

shifeiwen commented 9 months ago

Looking forward to progress