search

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.tech
MIT License
1.36k stars 86 forks source link

BitNet Support #98

Open xiongsp opened 2 weeks ago

xiongsp commented 2 weeks ago

Hello. Awesome project! Microsoft has published BitNet (https://github.com/microsoft/BitNet). Any plans to support it?

guinmoon commented 2 weeks ago

Hi. I haven't tested but these models should work.

xiongsp commented 2 weeks ago

Thanks! But it didn't work. bitnet_b1_58-3B.Q4_K_S.gguf didn't work correctly on my iPhone. It crashed with Load Model Error: [Done]. All setting templates have been tested. Hash of the model has been checked. Another model qwen2.5-3b-q5_0 works correctly.

guinmoon commented 2 weeks ago

Yeah, it's not working for me either. GreenSky in the description of the BitNet model suggests trying this model. It worked fine for me.