search

gokayfem / ComfyUI_VLM_nodes

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation
Apache License 2.0
409 stars 36 forks source link

GPU Offloading not Working - Cuda not detected #107

Open Iory1998 opened 1 month ago

Iory1998 commented 1 month ago

Llama.cpp seems to use CPU instead of the GPU (RTX3090) which makes the process very slow, No matter the number of GPU layers I set, the model will always be offloaded to the CPU. Also, It's seems that BLAS is activated but not used? image

doggeddalle commented 1 month ago

-_-

gokayfem commented 1 week ago

Yeah, even if you install it from llama-cpp-python library with CMAKE ARGS 3090 has this problem, i could not find a solution, i have 3090 too..

DavideAlidosi commented 4 days ago

I currently have two installations of ComfyUI, on the older one the GPU acceleration works, on the new one it does not. I did some research and noticed how the defect is not directly on Comfy but on some dependency, find where the real problem is though. I am also using a 3090, but my experience excludes a HW problem.