search

marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.
MIT License
1.8k stars 137 forks source link

Remove GGML_USE_CUBLAS when CT_HIPBLAS is defined #136

Open muaiyadh opened 1 year ago

muaiyadh commented 1 year ago

Hi,

I couldn't install the library with HIPBLAS because of missing CUDA stuff. Turns out there was an extra option for compiling with CUBLAS when CT_HIPBLAS is defined.

Also, should fix #134 and #135

bugfixin commented 1 year ago

Trying this myself, it seems to work, thank you!

CrashTD commented 1 year ago

Still does not build for me. However, the log is now significant shorter. https://gist.github.com/CrashTD/e3ef7d4901a22b7795e1618b196e3d36

maybe just a incompatibility with the still relatively new gfx1101?

bugfixin commented 1 year ago

Still does not build for me. However, the log is now significant shorter. https://gist.github.com/CrashTD/e3ef7d4901a22b7795e1618b196e3d36

maybe just a incompatibility with the still relatively new gfx1101?

It looks like your compilation is using the standard system compilers (cc and c++), try setting the environment variables CC="/opt/rocm/llvm/bin/clang" CXX="/opt/rocm/llvm/bin/clang++" (you can prepend that to the pip install command), e.g. CC="/opt/rocm/llvm/bin/clang" CXX="/opt/rocm/llvm/bin/clang++" CT_HIPBLAS=1 pip install ctransformers --no-binary ctransformers

CrashTD commented 1 year ago

It looks like your compilation is using the standard system compilers (cc and c++), ...

That in combination with the removal of the CUBLAS line fixed it. Thanks a lot!