Closed Fuckingnameless closed 5 months ago
Did you set --gpulayers? If yes, share the terminal output when the model is loaded
nope, i didn't set cause i wanted to use gpu only
If you want to get a generation speedup, you should offload layers to GPU.
If you want to get a generation speedup, you should offload layers to GPU.
i thought i'd read somewhere that if i don't specifiy it it should go all to the gpu but i was mistaken, thanks for the heads up, now trying to get a bit more performance on this old card
when i run the latest prebuilt binary with: ./koboldcpp-linux-x64 --usecublas
it uses the cpu even though i have the latest drivers/cuda, am i missing some flag or config?