LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.01k stars 350 forks source link

vulkan kinda messed up #989

Open Zuzia-Sweetheart opened 3 months ago

Zuzia-Sweetheart commented 3 months ago

i use a self compiled version of koboldcpp, the newest version

since i do not have AVX2 but i do have AVX1 i select vulkan old cpu, instead of showing me my nvidia quadro card it shows me ,,lvmpipe (LLVM 15.0.7 [there is also some stuff further but it cuts off])'', when i select clblast it does absolutely nothing

it was shown in the terminal: Vulkan0: llvmpipe (LLVM 15.0.7, 256 bits) (llvmpipe) | uma: 0 | fp16: 1 | warp size: 8 ggml_vulkan: Warning: Device type is CPU. This is probably not the device you want.

i use dell m4700 and i am on the newest version of linux mint, i do have proprietary nvidia drivers

LostRuins commented 2 months ago

Did you try changing selected GPU number? Is this the only GPU available? Also, what quadro card are you trying to use? Vulkan support might not be present in very old cards. Next version I will include avx1 cuda support.

Zuzia-Sweetheart commented 2 months ago

the 1 id gpu is an intel integrated gpu but it doesn't work for some reason

when i select id 2 it shows the llvmpipe thing, it technically works but kobold seems to struggle recognizing it as a gpu so it is slower than on failsafe mode

Zuzia-Sweetheart commented 2 months ago

and forgor to mention, its K2000M, if vulkan won't work, could you help out in making clblast work?

Zuzia-Sweetheart commented 2 months ago

@LostRuins

Zuzia-Sweetheart commented 2 months ago

@LostRuins and it is K2000M quadro, and if vulkan is not present, could you help out in making clblast wok?

and could nouvau driver be used with clblast?

LostRuins commented 2 months ago

That's a really old card. CLblast doesn't work for you?

Zuzia-Sweetheart commented 2 months ago

ye, unfortunately it doesn't

LostRuins commented 2 months ago

What error do you get? You might be stuck with CPU only speeds

Zuzia-Sweetheart commented 2 months ago

i promise i will answer soon, i have another mental episode