search

quic / efficient-transformers

This library empowers users to seamlessly port pretrained models and checkpoints on the HuggingFace (HF) hub (developed using HF transformers library) into inference-ready formats that run efficiently on Qualcomm Cloud AI 100 accelerators.
https://quic.github.io/efficient-transformers/
Other
47 stars 31 forks source link

Gorilla model working on AI100? #77

Closed willswordh closed 3 months ago

willswordh commented 3 months ago

Hi, is there any information that Gorilla OpenFunctions v2 will be able to run on AI100? Thanks!

ochougul commented 3 months ago

Hi @willswordh, Thanks for opening the issue. You should look at the architecture of the model in the config.json file on huggingface, If the architecture is same as one of the models written here -> https://github.com/quic/efficient-transformers?tab=readme-ov-file#validated-models, you can be sure that the new model will also be supported.

In your case, the Gorilla model is based on Llama architecture which is already supported.

ochougul commented 3 months ago

Closing this issue, please reopen if required in future.