search

Mobile-Artificial-Intelligence / maid

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.
MIT License
1.33k stars 133 forks source link

NPU Support ? #612

Open gformcreation opened 1 month ago

gformcreation commented 1 month ago

Hi Guys, Wanted to ask does this supports proper NPU ? for llm loading and processing?

danemadsen commented 1 month ago

No it doesn't as llama.cpp doesn't to the best of my knowledge. Once they support it, it will be supported here

gformcreation commented 1 month ago

FYI ... https://github.com/abetlen/llama-cpp-python/issues/1702