LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.34k stars 310 forks source link

Phi-3 vulkan? #931

Open paryska99 opened 1 week ago

paryska99 commented 1 week ago

Did anybody manage to make phi-3 work with vulkan backend? Im testing latest koboldcpp on a very restricted device when it comes to available drivers (windows steam-deck), so vulkan is the only backend that manages to work and be semi-quick. Codeqwen1.5 works great with vulkan on KCPP but phi-3 mini just crashes while loading.