search

Bip-Rep / sherpa

A mobile Implementation of llama.cpp
MIT License
293 stars 35 forks source link

Model file not working #14

Open abusaadp opened 1 year ago

abusaadp commented 1 year ago

Hi, Can you please send me the link of the model file which works in this project. All the model files I have downloaded are not working in this project.

Thanks

Tabiena commented 1 year ago

I'm using Orca mini 3B q4_0 (https://huggingface.co/TheBloke/orca_mini_3B-GGML/tree/main) with the apk from the fork of dsd (https://github.com/dsd/sherpa/releases/tag/2.2.1-dsd2) on a Samsung Note 10 lite (6GB).

abusaadp commented 1 year ago

I am getting this error error loading model: unknown (magic, version) combination: 67676a74, 00000003; is this really a GGML file? llama_init_from_file: failed to load model

I am using this file orca-mini-3b.ggmlv3.q4_0.bin

I am running this app in a Windows machine.

Tabiena commented 1 year ago

Sorry, I thought you were here for the android version. For Windows there is only the old version on this site, which probably only works with older ggmls. I would use Koboldcpp instead (https://github.com/LostRuins/koboldcpp).