I want to know how to enable AMD GPU or enable hipBLAS/ROCm on Windows. I tried the official HIP (sdk) and set the relevant environment flags, but it seems to have no effect. BLAS is always at 0. Is there any solution for Windows? Plan?

[chengjia604]

I want to know how to enable AMD GPU or enable hipBLAS/ROCm on Windows. I tried the official HIP (sdk) and set the relevant environment flags, but it seems to have no effect. BLAS is always at 0. Is there any solution for Windows? Plan?

AMD GPU: 6650xt windows11

[sorasoras]

There is no viable path for gpu like 6650XT on Windows for now. HIP SDK is not support for gfx1032. you need 6800XT or above. so you need to compile rocblas yourselves but that is not working right now as the vkpg source is not working.

[Meron0th]

Adding on instead of posting my own issue, because I think this is the same problem.

Here with a 7900xt, which is supported and should be working.

Expected Behavior

After building with CMAKE_ARGS="-DLLAMA_HIPBLAS=on" and FORCE_CMAKE=1, running a test script loads the model with BLAS = 1

Current Behavior

After building with CMAKE_ARGS="-DLLAMA_HIPBLAS=on" and FORCE_CMAKE=1, running a test script loads the model with BLAS = 0

Extracting w64devkit a and adding those manually to CMAKE_ARGS (as advised in the README) results in other errors, which I think are seperate issues (see additional troubleshooting)

End of verbose output from running a test file:

...................................................................................................
llama_new_context_with_model: n_ctx      = 4096
llama_new_context_with_model: freq_base  = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: KV self size  =  768.00 MiB, K (f16):  384.00 MiB, V (f16):  384.00 MiB
llama_build_graph: non-view tensors processed: 1012/1012
llama_new_context_with_model: compute buffer total size = 571.19 MiB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 |

Environment and Context

Windows 10.0.19045 7900 XT AMD HIP SDK installed Build with environmental variables: $env:CMAKE_ARGS="-DLLAMA_HIPBLAS=on, $env:FORCE_CMAKE=1

> python3 --version
Python 3.10.1
> make --version
GNU Make 4.4
Built for x86_64-w64-mingw32
> g++ --version
g++.exe (MinGW.org GCC-6.3.0-1) 6.3.0

Additional troubleshooting

I thought maybe it wasn't building hipBLAS at all, so I also set the environment variables $env:CMAKE_GENERATOR = "MinGW Makefiles" and CMAKE_ARGS="-DLLAMA_HIPBLAS=on -DCMAKE_C_COMPILER=C:/w64devkit/bin/gcc.exe -DCMAKE_CXX_COMPILER=C:/w64devkit/bin/g++.exe" Building this way fails with:

CMake Error at C:/Program Files/CMake/share/cmake-3.28/Modules/CMakeTestCCompiler.cmake:67 (message):
The C compiler

          "C:/w64devkit/bin/gcc.exe"

is not able to compile a simple test program.
It fails with the following output:

          Change Dir: '{user}/AppData/Local/Temp/tmpbbaghxaa/build/CMakeFiles/CMakeScratch/TryCompile-fyvzbc'

When I change the compilers to the path for Microsoft Visual Studio, I get the same error:

CMake Error at C:/Program Files/CMake/share/cmake-3.28/Modules/CMakeTestCCompiler.cmake:67 (message):
The C compiler

          "B:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe"

        is not able to compile a simple test program.

        It fails with the following output:

          Change Dir: '{user}/AppData/Local/Temp/tmp77se0tv8/build/CMakeFiles/CMakeScratch/TryCompile-lsyh3u'

I feel that the C Compiler failing is a seperate issue from this thread's issue of hipBLAS not activating, but it prevents trying to work around it.

Let me know what information can help troubleshoot, thank you