search

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.site
MIT License
1.05k stars 62 forks source link

Dolphin models problem (modelLoadError) #38

Closed BlobIsBack closed 2 months ago

BlobIsBack commented 5 months ago

They just doesn't seems to work, so far tested phi 2 dolphin and mistral 7b dolphin and it don't work. Maybe I'm just dumb, if that's the case can someone give me a step by step guide on how to use phi 2 dolphin ? (I'm using TestFlight version and iPhone XR)

BlobIsBack commented 5 months ago

Btw sorry for my bad English, I'm French

guinmoon commented 3 months ago

Can you tell me if this problem still occurs in the new version?

rd-neosoft commented 3 months ago

I was able to run phi2 on iphone 15 pro but not able to run the same model on iphone XR

BlobIsBack commented 3 months ago

Still the same problem

guinmoon commented 3 months ago

I was able to run phi2 on iphone 15 pro but not able to run the same model on iphone XR

Iphone XR only has 3GB RAM, so try using q3_k_s quantization, might be able to get it to run.