search

marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.
MIT License
1.79k stars 135 forks source link

CUDA error 35 at /home/runner/work/ctransformers/ctransformers/models/ggml/ggml-cuda.cu:5067: CUDA driver version is insufficient for CUDA runtime version #139

Open thistleknot opened 11 months ago

thistleknot commented 11 months ago

What cuda drivers are needed?

I'm able to run llama-cpp-pyhon and llama.cpp just fine, but not this tool

Ubuntu 22 Cuda 11.7 driver is 515.43.04

thistleknot commented 11 months ago

note works in 12.2 but not 11.7/11.8 :/ too bad soo sad for my 4GB VRAM cards

sujeendran commented 11 months ago

@thistleknot - Here is a fix that worked for me: https://github.com/marella/ctransformers/issues/90#issuecomment-1734127938

thistleknot commented 11 months ago

Thank you

I got llama.cpp to work on one card that Max's out on compute 5.2 by setting the compute capability explicitly in the makelist file. I presume this is similar. However I have yet to test similar changes on this. Will try sometime this week hopefully

On Mon, Sep 25, 2023, 9:59 AM Sujeendran Menon @.***> wrote:

@thistleknot https://github.com/thistleknot - Here is a fix that worked for me: #90 (comment) https://github.com/marella/ctransformers/issues/90#issuecomment-1734127938

— Reply to this email directly, view it on GitHub https://github.com/marella/ctransformers/issues/139#issuecomment-1734136815, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOUZU64EOXOPDJ42VTLX4GZ6HANCNFSM6AAAAAA45OK5KU . You are receiving this because you were mentioned.Message ID: @.***>

alifatmi commented 11 months ago

i also got this error but following commands worked for me.

Install CUDA libraries using:

pip install ctransformers[cuda] run above command and run your project code.