LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.33k stars 363 forks source link

Kobold fails to launch (ROCm) #1123

Closed EeduWave closed 2 months ago

EeduWave commented 2 months ago

I've been trying to use Kobold locally on my computer. However, every time I try to launch it the launching fails and KoboldCPP immediately closes/crashes. Trying to figure out what's happening is painful, since it takes an average of 4-5 minutes for Kobold to properly show the GUI and give me the ability to launch it - which then it proceeds to fail.

Here are screenshots of KoboldCPP seconds before it closes.

This one is what it shows when I use hipBLAS KAI01 (Portuguese text translation: [WinError 1114] A dynamic link library (DLL) initialization routine failed)

I've tried alternatively with the Vulkan option (probably unrelated, just did it to try if it changed anything), it instead gives me a different error KAI02

Here are my computer specs, in case it helps: Windows 10 Pro x64-based PC 22H2 Intel(R) Pentium(R) Gold G5400 CPU @ 3.70GHz / Intel64 Family 6 Model 158 Stepping 10 GenuineIntel ~3696 Mhz (AMD)

I have very little coding knowledge, so any help is welcome!

LostRuins commented 2 months ago

Hi,

The problem is that your CPU Intel(R) Pentium(R) Gold G5400 CPU does not support AVX or AVX2, thus it's unable to use any SIMD instructions.

Do you have a GPU? If so, what GPU is it?

Right now, the only option that will work is "Failsafe" CPU mode, you can try that. Also, the model you selected is likely going to be too slow for you, try this one: https://huggingface.co/concedo/KobbleTinyV2-1.1B-GGUF/resolve/main/KobbleTiny-Q4_K.gguf?download=true

EeduWave commented 2 months ago

My GPU is Radeon RX 570 Series.

Oh, and this is the error that appeared using the Failsafe mode.

KAI03

EeduWave commented 2 months ago

Wrong button, oops.

henk717 commented 2 months ago

The RX570 does not support ROCm, you would have to use Vulkan but you'd still be CPU limited.