LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.34k stars 310 forks source link

Add OpenMP support in CMakeList #939

Closed Nexesenex closed 1 week ago

Nexesenex commented 1 week ago

Useful for CPU based inference, but also for Cublas lowvram inference (TG)

See : https://github.com/ggerganov/llama.cpp/pull/7606