search

Mobile-Artificial-Intelligence / maid

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.
MIT License
1.03k stars 98 forks source link

qwen 2 model outputs gibberish. #581

Closed chaihahaha closed 4 days ago

chaihahaha commented 5 days ago

Screenshot_20240704_202921.jpg

qwen 2 1.5b and 7b quantized with Q6k or Q4km (by bartowski on huggingface) are not working on maid. The output is normal on PC with llama.cpp, but is weird on Maid App.

danemadsen commented 4 days ago

You might want to try this again now that ive published 1.2.9

chaihahaha commented 4 days ago

After upgrading to 1.2.9, qwen2 is working seamlessly and much faster, thank you very much for you awesome work！

danemadsen commented 4 days ago

@chaihahaha cool beans! :)