search

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.site
MIT License
1.06k stars 64 forks source link

Crashes on iPhone app via Testflight with Dolly v2 and Pythia model #7

Closed nehap27 closed 9 months ago

nehap27 commented 9 months ago

Hi, Thanks for the code. I downloaded the app via Testflight on my iPhone device (iPhone 14Pro, iOS 17). I tried to chat with Models Dolly v2 and Pythia, but the app crashed after asking 'How is the weather today?'. Attaching the chat and history json here. dolly-v2-3b-ggml_v3-q5_1_1695278118.txt

dolly-v2-3b-ggml_v3-q5_1_1695278118.txt

pythia-70m-ggml_v3-q5_1_1695277588.txt

guinmoon commented 9 months ago

Hi,

Thanks for the code. I downloaded the app via Testflight on my iPhone device (iPhone 14Pro, iOS 17).

I tried to chat with Models Dolly v2 and Pythia, but the app crashed after asking 'How is the weather today?'.

Attaching the chat and history json here.

dolly-v2-3b-ggml_v3-q5_1_1695278118.txt

dolly-v2-3b-ggml_v3-q5_1_1695278118.txt

pythia-70m-ggml_v3-q5_1_1695277588.txt

Based on what I saw in the files, you chose the wrong inference. dolly and pythia need to be run with gptneox inference not with llama