search

patientx / ComfyUI-Zluda

The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface. Now ZLUDA enhanced for better AMD GPU performance.
GNU General Public License v3.0
118 stars 8 forks source link

Error loading: caffe2_nvrtc.dll #7

Closed andharri closed 3 months ago

andharri commented 3 months ago

Trying to start ComfyUI on Windows 11 with AMD 7900 XTX GPU and I am seeing the following error:

Traceback (most recent call last): File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\main.py", line 73, in import comfy.utils File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\utils.py", line 1, in import torch File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch__init__.py", line 141, in raise err OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\lib\caffe2_nvrtc.dll" or one of its dependencies

The caffe2_nvrtc.dll file does exist in the location mentioned above.

I have the HIP SDK installed and the path values all look good. I did have an NVIDIA card in previously and used DDU to fully remove. Also cleaned out PIP cache

Worth noting that I have Automatic running with Zluda and all seems good there

Thanks

asilvamaia commented 3 months ago

I had the same erro. You added C:\Program Files\AMD\ROCm\5.7\bin in Path, on System Variable? Has 2 path, HIP_PATH and PATH.

andharri commented 3 months ago

In the system variables I have the following

Variable: HIP_PATH Value: C:\Program Files\AMD\ROCm\5.7\ In the Path variable I have an entry for %HIP_PATH%bin

andharri commented 3 months ago

If I add C:\Program Files\AMD\ROCm\5.7\bin to the Path variable explicitly I still have the same issue

patientx commented 3 months ago

After adding that to the path, restart pc, try it again and then , can you delete venv and re-run install bat, then share everything that happens there please ? Up to the error ?

andharri commented 3 months ago

I have explicitly added that entry to the path, rebooted and done a fresh install. It still gives the same error, here is the full output of install.bat

----------------------------------------------------- *************** COMFYUI-ZLUDA INSTALL *************** ----------------------------------------------------- *** Setting up the virtual enviroment ..................................................... *** Virtual enviroment activation ..................................................... *** Updating the pip package ..................................................... *** Installing required packages Collecting torch (from -r requirements.txt (line 1)) Using cached torch-2.3.0-cp310-cp310-win_amd64.whl.metadata (26 kB) Collecting torchsde (from -r requirements.txt (line 2)) Using cached torchsde-0.2.6-py3-none-any.whl.metadata (5.3 kB) Collecting torchvision (from -r requirements.txt (line 3)) Using cached torchvision-0.18.0-cp310-cp310-win_amd64.whl.metadata (6.6 kB) Collecting einops (from -r requirements.txt (line 4)) Using cached einops-0.8.0-py3-none-any.whl.metadata (12 kB) Collecting transformers>=4.25.1 (from -r requirements.txt (line 5)) Using cached transformers-4.41.2-py3-none-any.whl.metadata (43 kB) Collecting safetensors>=0.3.0 (from -r requirements.txt (line 6)) Using cached safetensors-0.4.3-cp310-none-win_amd64.whl.metadata (3.9 kB) Collecting aiohttp (from -r requirements.txt (line 7)) Using cached aiohttp-3.9.5-cp310-cp310-win_amd64.whl.metadata (7.7 kB) Collecting pyyaml (from -r requirements.txt (line 8)) Using cached PyYAML-6.0.1-cp310-cp310-win_amd64.whl.metadata (2.1 kB) Collecting Pillow (from -r requirements.txt (line 9)) Using cached pillow-10.3.0-cp310-cp310-win_amd64.whl.metadata (9.4 kB) Collecting scipy (from -r requirements.txt (line 10)) Using cached scipy-1.13.1-cp310-cp310-win_amd64.whl.metadata (60 kB) Collecting tqdm (from -r requirements.txt (line 11)) Using cached tqdm-4.66.4-py3-none-any.whl.metadata (57 kB) Collecting psutil (from -r requirements.txt (line 12)) Using cached psutil-5.9.8-cp37-abi3-win_amd64.whl.metadata (22 kB) Collecting kornia>=0.7.1 (from -r requirements.txt (line 15)) Using cached kornia-0.7.2-py2.py3-none-any.whl.metadata (12 kB) Collecting spandrel (from -r requirements.txt (line 16)) Using cached spandrel-0.3.4-py3-none-any.whl.metadata (14 kB) Collecting filelock (from torch->-r requirements.txt (line 1)) Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB) Collecting typing-extensions>=4.8.0 (from torch->-r requirements.txt (line 1)) Using cached typing_extensions-4.12.1-py3-none-any.whl.metadata (3.0 kB) Collecting sympy (from torch->-r requirements.txt (line 1)) Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch->-r requirements.txt (line 1)) Using cached networkx-3.3-py3-none-any.whl.metadata (5.1 kB) Collecting jinja2 (from torch->-r requirements.txt (line 1)) Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB) Collecting fsspec (from torch->-r requirements.txt (line 1)) Using cached fsspec-2024.5.0-py3-none-any.whl.metadata (11 kB) Collecting mkl<=2021.4.0,>=2021.1.1 (from torch->-r requirements.txt (line 1)) Using cached mkl-2021.4.0-py2.py3-none-win_amd64.whl.metadata (1.4 kB) Collecting numpy>=1.19 (from torchsde->-r requirements.txt (line 2)) Using cached numpy-1.26.4-cp310-cp310-win_amd64.whl.metadata (61 kB) Collecting trampoline>=0.1.2 (from torchsde->-r requirements.txt (line 2)) Using cached trampoline-0.1.2-py3-none-any.whl.metadata (10 kB) Collecting huggingface-hub<1.0,>=0.23.0 (from transformers>=4.25.1->-r requirements.txt (line 5)) Using cached huggingface_hub-0.23.2-py3-none-any.whl.metadata (12 kB) Collecting packaging>=20.0 (from transformers>=4.25.1->-r requirements.txt (line 5)) Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB) Collecting regex!=2019.12.17 (from transformers>=4.25.1->-r requirements.txt (line 5)) Using cached regex-2024.5.15-cp310-cp310-win_amd64.whl.metadata (41 kB) Collecting requests (from transformers>=4.25.1->-r requirements.txt (line 5)) Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB) Collecting tokenizers<0.20,>=0.19 (from transformers>=4.25.1->-r requirements.txt (line 5)) Using cached tokenizers-0.19.1-cp310-none-win_amd64.whl.metadata (6.9 kB) Collecting aiosignal>=1.1.2 (from aiohttp->-r requirements.txt (line 7)) Using cached aiosignal-1.3.1-py3-none-any.whl.metadata (4.0 kB) Collecting attrs>=17.3.0 (from aiohttp->-r requirements.txt (line 7)) Using cached attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB) Collecting frozenlist>=1.1.1 (from aiohttp->-r requirements.txt (line 7)) Using cached frozenlist-1.4.1-cp310-cp310-win_amd64.whl.metadata (12 kB) Collecting multidict<7.0,>=4.5 (from aiohttp->-r requirements.txt (line 7)) Using cached multidict-6.0.5-cp310-cp310-win_amd64.whl.metadata (4.3 kB) Collecting yarl<2.0,>=1.0 (from aiohttp->-r requirements.txt (line 7)) Using cached yarl-1.9.4-cp310-cp310-win_amd64.whl.metadata (32 kB) Collecting async-timeout<5.0,>=4.0 (from aiohttp->-r requirements.txt (line 7)) Using cached async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB) Collecting colorama (from tqdm->-r requirements.txt (line 11)) Using cached colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB) Collecting kornia-rs>=0.1.0 (from kornia>=0.7.1->-r requirements.txt (line 15)) Using cached kornia_rs-0.1.3-cp310-none-win_amd64.whl.metadata (8.9 kB) Collecting intel-openmp==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch->-r requirements.txt (line 1)) Using cached intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl.metadata (1.2 kB) Collecting tbb==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch->-r requirements.txt (line 1)) Using cached tbb-2021.12.0-py3-none-win_amd64.whl.metadata (1.1 kB) Collecting idna>=2.0 (from yarl<2.0,>=1.0->aiohttp->-r requirements.txt (line 7)) Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB) Collecting MarkupSafe>=2.0 (from jinja2->torch->-r requirements.txt (line 1)) Using cached MarkupSafe-2.1.5-cp310-cp310-win_amd64.whl.metadata (3.1 kB) Collecting charset-normalizer<4,>=2 (from requests->transformers>=4.25.1->-r requirements.txt (line 5)) Using cached charset_normalizer-3.3.2-cp310-cp310-win_amd64.whl.metadata (34 kB) Collecting urllib3<3,>=1.21.1 (from requests->transformers>=4.25.1->-r requirements.txt (line 5)) Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) Collecting certifi>=2017.4.17 (from requests->transformers>=4.25.1->-r requirements.txt (line 5)) Using cached certifi-2024.6.2-py3-none-any.whl.metadata (2.2 kB) Collecting mpmath<1.4.0,>=1.1.0 (from sympy->torch->-r requirements.txt (line 1)) Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Using cached torch-2.3.0-cp310-cp310-win_amd64.whl (159.8 MB) Using cached torchsde-0.2.6-py3-none-any.whl (61 kB) Using cached torchvision-0.18.0-cp310-cp310-win_amd64.whl (1.2 MB) Using cached einops-0.8.0-py3-none-any.whl (43 kB) Using cached transformers-4.41.2-py3-none-any.whl (9.1 MB) Using cached safetensors-0.4.3-cp310-none-win_amd64.whl (287 kB) Using cached aiohttp-3.9.5-cp310-cp310-win_amd64.whl (370 kB) Using cached PyYAML-6.0.1-cp310-cp310-win_amd64.whl (145 kB) Using cached pillow-10.3.0-cp310-cp310-win_amd64.whl (2.5 MB) Using cached scipy-1.13.1-cp310-cp310-win_amd64.whl (46.2 MB) Using cached tqdm-4.66.4-py3-none-any.whl (78 kB) Using cached psutil-5.9.8-cp37-abi3-win_amd64.whl (255 kB) Using cached kornia-0.7.2-py2.py3-none-any.whl (825 kB) Using cached spandrel-0.3.4-py3-none-any.whl (268 kB) Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Using cached async_timeout-4.0.3-py3-none-any.whl (5.7 kB) Using cached attrs-23.2.0-py3-none-any.whl (60 kB) Using cached frozenlist-1.4.1-cp310-cp310-win_amd64.whl (50 kB) Using cached huggingface_hub-0.23.2-py3-none-any.whl (401 kB) Using cached fsspec-2024.5.0-py3-none-any.whl (316 kB) Using cached kornia_rs-0.1.3-cp310-none-win_amd64.whl (1.2 MB) Using cached mkl-2021.4.0-py2.py3-none-win_amd64.whl (228.5 MB) Using cached intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl (3.5 MB) Using cached tbb-2021.12.0-py3-none-win_amd64.whl (286 kB) Using cached multidict-6.0.5-cp310-cp310-win_amd64.whl (28 kB) Using cached numpy-1.26.4-cp310-cp310-win_amd64.whl (15.8 MB) Using cached packaging-24.0-py3-none-any.whl (53 kB) Using cached regex-2024.5.15-cp310-cp310-win_amd64.whl (268 kB) Using cached tokenizers-0.19.1-cp310-none-win_amd64.whl (2.2 MB) Using cached trampoline-0.1.2-py3-none-any.whl (5.2 kB) Using cached typing_extensions-4.12.1-py3-none-any.whl (37 kB) Using cached yarl-1.9.4-cp310-cp310-win_amd64.whl (76 kB) Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB) Using cached filelock-3.14.0-py3-none-any.whl (12 kB) Using cached jinja2-3.1.4-py3-none-any.whl (133 kB) Using cached networkx-3.3-py3-none-any.whl (1.7 MB) Using cached requests-2.32.3-py3-none-any.whl (64 kB) Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB) Using cached certifi-2024.6.2-py3-none-any.whl (164 kB) Using cached charset_normalizer-3.3.2-cp310-cp310-win_amd64.whl (100 kB) Using cached idna-3.7-py3-none-any.whl (66 kB) Using cached MarkupSafe-2.1.5-cp310-cp310-win_amd64.whl (17 kB) Using cached mpmath-1.3.0-py3-none-any.whl (536 kB) Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) Installing collected packages: trampoline, tbb, mpmath, intel-openmp, urllib3, typing-extensions, sympy, safetensors, regex, pyyaml, psutil, Pillow, packaging, numpy, networkx, multidict, mkl, MarkupSafe, kornia-rs, idna, fsspec, frozenlist, filelock, einops, colorama, charset-normalizer, certifi, attrs, async-timeout, yarl, tqdm, scipy, requests, jinja2, aiosignal, torch, huggingface-hub, aiohttp, torchvision, torchsde, tokenizers, kornia, transformers, spandrel Successfully installed MarkupSafe-2.1.5 Pillow-10.3.0 aiohttp-3.9.5 aiosignal-1.3.1 async-timeout-4.0.3 attrs-23.2.0 certifi-2024.6.2 charset-normalizer-3.3.2 colorama-0.4.6 einops-0.8.0 filelock-3.14.0 frozenlist-1.4.1 fsspec-2024.5.0 huggingface-hub-0.23.2 idna-3.7 intel-openmp-2021.4.0 jinja2-3.1.4 kornia-0.7.2 kornia-rs-0.1.3 mkl-2021.4.0 mpmath-1.3.0 multidict-6.0.5 networkx-3.3 numpy-1.26.4 packaging-24.0 psutil-5.9.8 pyyaml-6.0.1 regex-2024.5.15 requests-2.32.3 safetensors-0.4.3 scipy-1.13.1 spandrel-0.3.4 sympy-1.12.1 tbb-2021.12.0 tokenizers-0.19.1 torch-2.3.0 torchsde-0.2.6 torchvision-0.18.0 tqdm-4.66.4 trampoline-0.1.2 transformers-4.41.2 typing-extensions-4.12.1 urllib3-2.2.1 yarl-1.9.4 ..................................................... *** Installing torch for AMD GPUs (First file is 2.7 GB, please be patient) Found existing installation: torch 2.3.0 Uninstalling torch-2.3.0: Successfully uninstalled torch-2.3.0 Found existing installation: torchvision 0.18.0 Uninstalling torchvision-0.18.0: Successfully uninstalled torchvision-0.18.0 Looking in indexes: https://download.pytorch.org/whl/cu118 Collecting torch==2.3.0 Using cached https://download.pytorch.org/whl/cu118/torch-2.3.0%2Bcu118-cp310-cp310-win_amd64.whl (2673.0 MB) Collecting torchvision Using cached https://download.pytorch.org/whl/cu118/torchvision-0.18.0%2Bcu118-cp310-cp310-win_amd64.whl (4.9 MB) Requirement already satisfied: filelock in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (3.14.0) Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (4.12.1) Requirement already satisfied: sympy in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (1.12.1) Requirement already satisfied: networkx in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (3.3) Requirement already satisfied: jinja2 in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (3.1.4) Requirement already satisfied: fsspec in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (2024.5.0) Requirement already satisfied: mkl<=2021.4.0,>=2021.1.1 in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torch==2.3.0) (2021.4.0) Requirement already satisfied: numpy in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torchvision) (1.26.4) Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from torchvision) (10.3.0) Requirement already satisfied: intel-openmp==2021.* in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from mkl<=2021.4.0,>=2021.1.1->torch==2.3.0) (2021.4.0) Requirement already satisfied: tbb==2021.* in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from mkl<=2021.4.0,>=2021.1.1->torch==2.3.0) (2021.12.0) Requirement already satisfied: MarkupSafe>=2.0 in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from jinja2->torch==2.3.0) (2.1.5) Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in c:\users\andy\desktop\sd\comfyui-zluda\venv\lib\site-packages (from sympy->torch==2.3.0) (1.3.0) Installing collected packages: torch, torchvision Successfully installed torch-2.3.0+cu118 torchvision-0.18.0+cu118 ..................................................... *** Installing Comfyui Manager Cloning into 'ComfyUI-Manager'... remote: Enumerating objects: 9986, done. remote: Counting objects: 100% (228/228), done. remote: Compressing objects: 100% (82/82), done. remote: Total 9986 (delta 161), reused 194 (delta 146), pack-reused 9758 Receiving objects: 100% (9986/9986), 6.79 MiB | 33.58 MiB/s, done. Resolving deltas: 100% (7231/7231), done. ..................................................... *** Installing ComfyUI-Impact-Pack Cloning into 'ComfyUI-Impact-Pack'... remote: Enumerating objects: 4183, done. remote: Counting objects: 100% (1726/1726), done. remote: Compressing objects: 100% (551/551), done. remote: Total 4183 (delta 1253), reused 1580 (delta 1131), pack-reused 2457 Receiving objects: 100% (4183/4183), 31.74 MiB | 31.87 MiB/s, done. Resolving deltas: 100% (2795/2795), done. Cloning into 'impact_subpack'... remote: Enumerating objects: 102, done. remote: Counting objects: 100% (102/102), done. remote: Compressing objects: 100% (76/76), done. remote: Total 102 (delta 46), reused 70 (delta 21), pack-reused 0 Receiving objects: 100% (102/102), 25.96 KiB | 279.00 KiB/s, done. Resolving deltas: 100% (46/46), done. ..................................................... *** "Patching ZLUDA" The system cannot find the file specified. 1 file(s) copied. 1 file(s) copied. 1 file(s) copied. *ZLUDA is patched. * ..................................................... *** Installation is done. You can use "start.bat" to start the app later. ..................................................... *** For now app is going to start for the first time. ## ComfyUI-Manager: installing dependencies. (GitPython) ## ComfyUI-Manager: installing dependencies done. ** ComfyUI startup time: 2024-06-02 23:15:21.074356 ** Platform: Windows ** Python version: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] ** Python executable: C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\Scripts\python.exe ** Log path: C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfyui.log Prestartup times for custom nodes: 9.0 seconds: C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\custom_nodes\ComfyUI-Manager Traceback (most recent call last): File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\main.py", line 73, in import comfy.utils File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\utils.py", line 1, in import torch File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\__init__.py", line 141, in raise err OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\lib\caffe2_nvrtc.dll" or one of its dependencies. (venv) C:\Users\Andy\Desktop\SD\ComfyUI-Zluda>
patientx commented 3 months ago

I have explicitly added that entry to the path, rebooted and done a fresh install. It still gives the same error, here is the full output of install.bat

First , is sdnext also on your desktop, I think installing stuff directly under "desktop" might cause some problems Secondly, since you are using sdnext -with zluda?- .. try this : Open patchzluda.bat with notepad etc and change the third line ::

curl -s -L https://github.com/lshqqytiger/ZLUDA/releases/download/rel.2804604c29b5fa36deca9ece219d3970b61d4c27/ZLUDA-windows-amd64.zip > zluda.zip

to

curl -s -L https://github.com/lshqqytiger/ZLUDA/releases/download/rel.11cc5844514f93161e0e74387f04e2c537705a82/ZLUDA-windows-amd64.zip > zluda.zip

save it , run patchzluda.bat

then to be absolutely sure , when inside comfyui-zluda folder, in cmdline run

venv\scripts\activate after that you should be in (venv) .. run

pip show torch it should be 2.3.0+cu118

make sure these three files are inside "ComfyUI-Zluda\venv\Lib\site-packages\torch\lib" cublas64_11.dll (196kb) cusparse64_11.dll (193kb) nvrtc64_112_0.dll (125kb) . Sizes might be a bit different since we just changed the zluda version but not in 10s of mb's just around these kb sizes.

also run this from commandline anywhere ::

hipinfo

it should output some info about your amd gpu.

if even after these you are still having the same problem, nothing else comes to mind other than maybe some nvidia dll left in your windows (you can use ddu to be sure also for the last ditch effort, make sure you don't have any NVIDIA driver dll files. under C:\Windows\System32. )

Because of these problems occuring sdnext changed the way it handles zluda I think, I don't have their expertise but for most people these solutions work. Daily have 15-20 new installs and your problem with the caffee dll stuff was first in weeks.

Still even after these if you want to try something else or take a look it deeply, if you have discord and in sdnext discord, please write about this under https://discord.com/channels/1101998836328697867/1246589342105866280 .(this is under COMMUNITY-RESOURCES.) (not the main zluda help channel !)

andharri commented 3 months ago

Hi,

So I have tried both versions of zluda and still see the same issues.

pip show torch is giving me 2.3.0+cu118

The DLL files look good size wise

I think my PC just hates me!

Do you have another link to that Discord channel, the one posted is showing as Disco app is showing no channels there

Thanks for all your help already!

patientx commented 3 months ago

add : patientx#5924

Hi,

So I have tried both versions of zluda and still see the same issues.

pip show torch is giving me 2.3.0+cu118

The DLL files look good size wise

I think my PC just hates me!

Do you have another link to that Discord channel, the one posted is showing as Disco app is showing no channels there

Thanks for all your help already!

add : patientx#5924

andharri commented 3 months ago

If I copy the entire venvdirectory from automaticto comfyui-zluda it is able to start, but as soon as I start an image, I get the following errors, not sure if this is useful or not:

!!! Exception during processing!!! CUDA error: operation not supported CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. Traceback (most recent call last): File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\nodes.py", line 58, in encode cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\sd.py", line 134, in encode_from_tokens self.load_model() File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\sd.py", line 154, in load_model model_management.load_model_gpu(self.patcher) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\model_management.py", line 485, in load_model_gpu return load_models_gpu([model]) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\model_management.py", line 479, in load_models_gpu cur_loaded_model = loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\model_management.py", line 324, in model_load raise e File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\model_management.py", line 320, in model_load self.real_model = self.model.patch_model(device_to=patch_model_to, patch_weights=load_weights) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\comfy\model_patcher.py", line 281, in patch_model self.model.to(device_to) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\nn\modules\module.py", line 1173, in to return self._apply(convert) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\nn\modules\module.py", line 779, in _apply module._apply(fn) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\nn\modules\module.py", line 779, in _apply module._apply(fn) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\nn\modules\module.py", line 779, in _apply module._apply(fn) [Previous line repeated 2 more times] File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\nn\modules\module.py", line 804, in _apply param_applied = fn(param) File "C:\Users\Andy\Desktop\SD\ComfyUI-Zluda\venv\lib\site-packages\torch\nn\modules\module.py", line 1159, in convert return t.to( RuntimeError: CUDA error: operation not supported CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

If i re-run patchzludait fails again with the original error from the start

andharri commented 3 months ago

Still not managed to resolve the issue, so for now going to use an old Nvidia card :(

skarr commented 3 months ago

@andharri I've seen similar issues related to python versions from Microsoft store, specifically version 3.10.

  1. Uninstall MS python release (or at the very least remove it from PATH)
  2. Install latest 3.11 release from python.org
  3. Make sure it's added to the PATH and prioritized over other python versions where python and python --version.
  4. Remove or rename venv folder in ComfyUI root dir
  5. Run install.bat again
  6. After ComfyUI web page comes up your first prompt will take 10 to 15min

Good luck.

andharri commented 3 months ago

Hi,

That has worked! Thank you so much for the help.

Thanks to @patientx as well for running the repo and the help too

patientx commented 3 months ago

Good to know there is a solution ! Have to add this to troubleshooting now :)

patientx commented 3 months ago

@andharri I've seen similar issues related to python versions from Microsoft store, specifically version 3.10.

  1. Uninstall MS python release (or at the very least remove it from PATH)
  2. Install latest 3.11 release from python.org
  3. Make sure it's added to the PATH and prioritized over other python versions where python and python --version.
  4. Remove or rename venv folder in ComfyUI root dir
  5. Run install.bat again
  6. After ComfyUI web page comes up your first prompt will take 10 to 15min
  • 3.11 is just an arbitrary version that I tried just to get it to work, other versions might be faster
  • Zluda generates a file in the following dir C:\Users\AppData\Local\ZLUDA\ComputeCache\ the first time you run it. You can check your resource monitor to see what files python is accessing, just be patient with this one. Get coffee, watch something, come back, get more coffee.

Good luck.

is it about the store version or 3.10 itself ? Because didn't see many problems like this in a while, normal 3.10 probably enough, what do you think ? / don't want to change base at this point for the repo /

skarr commented 3 months ago

is it about the store version or 3.10 itself ? Because didn't see many problems like this in a while, normal 3.10 probably enough, what do you think ? / don't want to change base at this point for the repo /

I tested it today. It's just the Microsoft store version that's causing problems.

Version 3.10.9 works when I used the installer from python.org

patientx commented 3 months ago

is it about the store version or 3.10 itself ? Because didn't see many problems like this in a while, normal 3.10 probably enough, what do you think ? / don't want to change base at this point for the repo /

I tested it today. It's just the Microsoft store version that's causing problems.

Version 3.10.9 works when I used the installer from python.org

Thanks ! Added the warning to the readme.