Closed pHaeusler closed 7 months ago
Yes. We have updated weights of MobileLLaMA.
Is there any android app that uses this ? Sherpa (llama2 7b) run slowly, and even crash with input/long context with 12gb (+4gb swap) on SD865 ...
Edit : Apparently there's this more updated fork of Sherpa https://github.com/Mobile-Artificial-Intelligence/maid/
Is there any android app that uses this ? Sherpa (llama2 7b) run slowly, and even crash with input/long context with 12gb (+4gb swap) on SD865 ...
Edit : Apparently there's this more updated fork of Sherpa https://github.com/Mobile-Artificial-Intelligence/maid/
Based on our experience, it is normal for the 7B model to have slow inference on mobile devices. We suggest reducing the model size or using higher specification equipment.
Awesome work!
Do you plan on releasing the weights for MobileVLM?