Feature Suggestion: CoreML/Neural engine

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.

https://llmfarm.site

MIT License

1.05k stars 62 forks source link

Feature Suggestion: CoreML/Neural engine #69

Closed EnderRobber101 closed 3 weeks ago

EnderRobber101 commented 4 weeks ago

Have you considered using CoreML to access the neural engine? I thought it would boost speeds. Thanks!

guinmoon commented 3 weeks ago

llama.cpp only uses Metal at the moment. And LLMFarm is completely based on llama.cpp. The reasons why the Neural engine is not used can be found here.