search

nktice / AMD-AI

AMD (Radeon GPU) ROCm based setup for popular AI tools on Ubuntu 22.04 / 23.04 / 23.10 / 24.04
Other
134 stars 8 forks source link

llama.cpp advice for: gfx1032 #6

Open mikeperalta1 opened 2 weeks ago

mikeperalta1 commented 2 weeks ago

Any advice for an older 6600XT card that seems to want "gfx1032" for llama.cpp? I've tried with Ubuntu 24 and 23.10 and get this crash:


rocBLAS error: Cannot read /opt/rocm/lib/rocblas/library/TensileLibrary.dat: Illegal seek for GPU arch : gfx1032
 List of available TensileLibrary Files : 
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1101.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx940.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1100.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1030.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx900.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx908.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx90a.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1102.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx941.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx906.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx942.dat"
/home/llm/bin/chat: line 22: 10172 Aborted                 (core dumped) /home/llm/opt/llama.cpp/main --model "/home/llm/opt/llama.cpp/models/llama-2-7b-chat.Q2_K.gguf" --interactive --color -n 1024 --prompt ""
nktice commented 2 weeks ago

When Mixtral came out I composed these instructions to get it to work - it includes details about llama.cpp such as compiling - https://github.com/nktice/AMD-AI/blob/main/Mixtral.md - I am wondering if you've tried such an install on your own system? Custom fresh install may allow for compiling that supports your card.

mikeperalta1 commented 2 weeks ago

Hey I appreciate you getting back to me. I ended up solving the above error with an environment trick:

HSA_OVERRIDE_GFX_VERSION=10.3.0

llama.cpp still failed to utilize my card but at least it didn't crash at that point. Ended up jumping ship to ollama which worked right away.

nktice commented 1 week ago

Someone else posted about llama-cpp in another thread - https://github.com/nktice/AMD-AI/issues/7 So I figured out a new way to install it, and explained that there.

mikeperalta1 commented 1 week ago

Nice, I'll check it out!