search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.66k stars 334 forks source link

[ROCm fork]: Errors loading or using models #692

Closed brknsoul closed 5 months ago

brknsoul commented 6 months ago

I can't seem to load any models with the ROCm fork on my 6600xt.

With "Use QuantMalMut (mmq)" ticked, I get;

CUDA error: named symbol not found
  current device: 0, in function ggml_cuda_op_mul_mat at D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml-cuda.cu:9464
  hipGetLastError()
GGML_ASSERT: D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml-cuda.cu:241: !"CUDA error"

Without mmq ticked, the browser launches, but any attempt at generation results in nonsense;

image

BrandenStoberReal commented 5 months ago

Can confirm this is also happening to me on Ubuntu 23.10 with Python 3.11.8.

brknsoul commented 5 months ago

This issue is solved with later releases of kcpp_rocm (at least for windows).