MrForExample / ComfyUI-3D-Pack

An extensive node suite that enables ComfyUI to process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc.)
MIT License
2.41k stars 245 forks source link

ImportError: DLL load failed while importing _C #151

Open Scharwunzel opened 6 months ago

Scharwunzel commented 6 months ago

Hi, when I start ConfyUI I get the following error in the (German) CMD: Traceback (most recent call last): File "C:\ComfyUI\ComfyUI\nodes.py", line 1879, in load_custom_node module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "C:\ComfyUI\ComfyUI\custom_nodes\ComfyUI-3D-Pack__init.py", line 28, in module = importlib.import_module(f".{nodes_filename}", package=name) File "importlib\init.py", line 126, in import_module File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "C:\ComfyUI\ComfyUI\custom_nodes\ComfyUI-3D-Pack\nodes.py", line 33, in from .algorithms.main_3DGS import GaussianSplatting, GaussianSplattingCameraController, GSParams File "C:\ComfyUI\ComfyUI\custom_nodes\ComfyUI-3D-Pack\algorithms\main_3DGS.py", line 12, in from .main_3DGS_renderer import GaussianSplattingRenderer File "C:\ComfyUI\ComfyUI\custom_nodes\ComfyUI-3D-Pack\algorithms\main_3DGS_renderer.py", line 24, in from diff_gaussian_rasterization import ( File "C:\ComfyUI\python_embeded\lib\site-packages\diff_gaussian_rasterization\init__.py", line 15, in from . import _C ImportError: DLL load failed while importing _C: Das angegebene Modul wurde nicht gefunden.

I already found out from the threat https://github.com/MrForExample/ComfyUI-3D-Pack/issues/77 that the error for others was due to the fact that they didn't install Visual Studio with C++, but I had this before installing Confyui and can't find a missing build either. Can someone maybe help me with what else could be the problem or where else I could look? Thank you in advance for any help!

YanWenKun commented 6 months ago

In my tests I found several DLL load failed while importing _C cases and it was simply fixed by downgrading PyTorch to 2.2.

I guess core packages like PyTorch3D are not yet ready for PyTorch 2.3.

zcfire2017 commented 6 months ago

@Scharwunzel
image Check if your cuda version is 12.1. Torch3d currently only supports at most Ao 12.1. Open cmd and enter "nvcc -- version". If there is cuda information, it means the driver has been installed. Check if the version is 12.1. If not, you need to reinstall cuda. After reinstalling, you need to run the installation command of ComfyUI-3D-Pack again

Scharwunzel commented 6 months ago

hi, sorry for the late reply and thank you für your help.

I have now tried both tips, unfortunately without success.

I have set PyTorch to "Version: 2.2.2+cu121"

The cuda version is also available at "Cuda compilation tools, release 12.1, V12.1.66 Build cuda_12.1.r12.1/compiler.32415258_0"

Both is showing while versions check in the cmd under \ComfyUI\python_embeded.

At start of Comfy there are following versions: "Current version of llama_cpp: 0.2.26+cu118 Current version of timm: 0.9.12 [ReActor] - STATUS - Running v0.5.0-b3 in ComfyUI Torch version: 2.3.0+cu118"

Do I have to do anything special for this to be taken over?

ColaCheese commented 6 months ago

Same problem here

YanWenKun commented 6 months ago

@Scharwunzel Try force-reinstall:

python.exe -s -m pip install --force-reinstall xformers==0.0.25.post1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 --extra-index-url https://pypi.org/simple

(By xformers==0.0.25.post1 we imply using PyTorch 2.2.2)

Scharwunzel commented 6 months ago

@YanWenKun I get this Message with this command:

Screenshot 2024-05-24 002840
zpengcom commented 6 months ago

image

image

zpengcom commented 6 months ago

image

image

@Scharwunzel Try force-reinstall:

python.exe -s -m pip install --force-reinstall xformers==0.0.25.post1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 --extra-index-url https://pypi.org/simple

(By xformers==0.0.25.post1 we imply using PyTorch 2.2.2)

It works for me. .🥰

YanWenKun commented 6 months ago

@Scharwunzel This error message was about other dependencies, you can ignore it, or:

python.exe -s -m pip install --force-reinstall pillow==9.5.0 numpy==1.25.2 cryptography
midnightrebels commented 5 months ago

Desktop Screenshot 2024 06 10 - 00 47 05 01

I am able to import 3D pack successfully however there's a few stuff that I don't understand. When force reinstalling xformers==0.0.25.post1, some lines shows "not on path". Is that why I'm seeing this future warning?

YanWenKun commented 5 months ago

@midnightrebels The warning in the screenshot is telling "some of the code is going to be outdated", and you can safely ignore it.

GPU-server commented 1 month ago

Hello @YanWenKun , is the 2.2.2 version still usable for this solution? I guess a new solution need to be updated for 2.4.0? Ho can I solve the problem (ImportError: DLL load failed while importing _C: The specified module could not be found.) in that case?

YanWenKun commented 1 month ago

@GPU-server

1. Referring to https://github.com/YanWenKun/ComfyUI-Windows-Portable/issues/7 , just use 2.2.2 with old commit.

2. Or if you want to update to the latest 3D-Pack, you need PyTorch 2.4 or 2.3.

python.exe -s -m pip install --force-reinstall xformers==0.0.27.post2 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 --extra-index-url https://pypi.org/simple

Then update/reinstall 3D-Pack.

3. Or just use all-in-one package: https://github.com/YanWenKun/ComfyUI-Windows-Portable https://github.com/YanWenKun/ComfyUI-Docker/tree/main/comfy3d-pt23

GPU-server commented 1 month ago

Hi thanks for your answer @YanWenKun before running your command, I would like to show you my current set and tell if you think I should proceed: I have this:

Package Version


accelerate 1.0.0 aiohappyeyeballs 2.4.0 aiohttp 3.10.5 aiosignal 1.3.1 antlr4-python3-runtime 4.9.3 attrs 24.2.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 cmake 3.30.4 colorama 0.4.6 coloredlogs 15.0.1 contourpy 1.3.0 cryptography 43.0.1 cycler 0.12.1 dataclasses-json 0.6.7 Deprecated 1.2.14 diff_gaussian_rasterization 0.0.0 diffusers 0.30.3 einops 0.8.0 filelock 3.15.4 flatbuffers 24.3.25 fonttools 4.54.1 frozenlist 1.4.1 fsspec 2024.9.0 ftfy 6.2.3 fvcore 0.1.5.post20221221 gitdb 4.0.11 GitPython 3.1.43 gpytoolbox 0.3.2 hatchling 1.25.0 huggingface-hub 0.24.6 humanfriendly 10.0 idna 3.8 imageio 2.35.1 imageio-ffmpeg 0.5.1 importlib_metadata 8.5.0 iopath 0.1.10 jaxtyping 0.2.34 Jinja2 3.1.4 jsonschema 4.23.0 jsonschema-specifications 2024.10.1 kiui 0.2.13 kiwisolver 1.4.7 kornia 0.7.3 kornia_rs 0.1.5 lazy_loader 0.4 lightning-utilities 0.11.7 llvmlite 0.43.0 markdown-it-py 3.0.0 MarkupSafe 2.1.5 marshmallow 3.22.0 matplotlib 3.9.2 matrix-client 0.4.0 mdurl 0.1.2 mpmath 1.3.0 msvc_runtime 14.40.33807 multidict 6.0.5 mypy-extensions 1.0.0 nerfacc 0.5.3 networkx 3.3 ninja 1.11.1.1 numba 0.60.0 numpy 1.26.4 nvdiffrast 0.3.1 omegaconf 2.3.0 onnxruntime 1.19.2 open_clip_torch 2.26.1 opencv-python 4.10.0.84 opencv-python-headless 4.10.0.84 packaging 24.1 pathspec 0.12.1 peft 0.13.1 pillow 10.4.0 pip 24.2 platformdirs 4.3.6 pluggy 1.5.0 plyfile 1.1 pointnet2_ops 3.0.0 pooch 1.8.2 portalocker 2.10.1 protobuf 5.28.2 psutil 6.0.0 pycparser 2.22 pygit2 1.15.1 PyGithub 2.4.0 pygltflib 1.16.2 Pygments 2.18.0 pyhocon 0.3.61 PyJWT 2.9.0 PyMatting 1.1.12 PyMCubes 0.1.6 pymeshlab 2023.12.post2 PyNaCl 1.5.0 pyparsing 3.1.4 pyreadline3 3.5.4 python-dateutil 2.9.0.post0 pytorch-lightning 2.4.0 pytorch-msssim 1.0.0 pytorch3d 0.7.8 pywin32 307 PyYAML 6.0.2 referencing 0.35.1 regex 2024.7.24 rembg 2.0.59 requests 2.32.3 rich 13.9.2 rpds-py 0.20.0 safetensors 0.4.5 scikit-image 0.24.0 scipy 1.14.1 sentencepiece 0.2.0 setuptools 74.1.2 shellingham 1.5.4 simple_knn 0.0.0 six 1.16.0 slangtorch 1.2.6 smmap 5.0.1 soundfile 0.12.1 spandrel 0.3.4 sympy 1.13.2 tabulate 0.9.0 termcolor 2.5.0 tifffile 2024.9.20 timm 1.0.9 tokenizers 0.19.1 torch 2.4.0+cu118 torch_scatter 2.1.2+pt24cu118 torchaudio 2.4.0 torchmetrics 1.4.2 torchsde 0.2.6 torchtyping 0.1.5 torchvision 0.19.0+cu118 tqdm 4.66.5 trampoline 0.1.2 transformers 4.44.2 trimesh 4.4.9 trove-classifiers 2024.9.12 typeguard 2.13.3 typer 0.12.5 typing_extensions 4.12.2 typing-inspect 0.9.0 urllib3 1.26.20 wcwidth 0.2.13 wheel 0.44.0 wrapt 1.16.0 xatlas 0.0.9 xformers 0.0.27.post2+cu118 yacs 0.1.8 yarl 1.9.11 zipp 3.20.2

And I have cuda toolkit 12.1 run install.py multiple times Put security level to weak to let comfy manager "try" to fix the issues. And even did a quick "update_comfyui.bat". It solved an issue related to pytorch audio was not being able to be set or something. I remained with the :

...init.py", line 53, in module = importlib.import_module(f".{nodes_filename}", package=name) ImportError: DLL load failed while importing _C: The specified module could not be found.

I am on the right path?

GPU-server commented 1 month ago

OK @YanWenKun Just tried your line: python.exe -s -m pip install --force-reinstall xformers==0.0.27.post2 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 --extra-index-url https://pypi.org/simple Went from that list (pip) to this one:

Package Version


accelerate 1.0.0 aiohappyeyeballs 2.4.0 aiohttp 3.10.5 aiosignal 1.3.1 antlr4-python3-runtime 4.9.3 attrs 24.2.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 cmake 3.30.4 colorama 0.4.6 coloredlogs 15.0.1 contourpy 1.3.0 cryptography 43.0.1 cycler 0.12.1 dataclasses-json 0.6.7 Deprecated 1.2.14 diff_gaussian_rasterization 0.0.0 diffusers 0.30.3 einops 0.8.0 filelock 3.16.1 flatbuffers 24.3.25 fonttools 4.54.1 frozenlist 1.4.1 fsspec 2024.9.0 ftfy 6.2.3 fvcore 0.1.5.post20221221 gitdb 4.0.11 GitPython 3.1.43 gpytoolbox 0.3.2 hatchling 1.25.0 huggingface-hub 0.24.6 humanfriendly 10.0 idna 3.8 imageio 2.35.1 imageio-ffmpeg 0.5.1 importlib_metadata 8.5.0 iopath 0.1.10 jaxtyping 0.2.34 Jinja2 3.1.4 jsonschema 4.23.0 jsonschema-specifications 2024.10.1 kiui 0.2.13 kiwisolver 1.4.7 kornia 0.7.3 kornia_rs 0.1.5 lazy_loader 0.4 lightning-utilities 0.11.7 llvmlite 0.43.0 markdown-it-py 3.0.0 MarkupSafe 3.0.1 marshmallow 3.22.0 matplotlib 3.9.2 matrix-client 0.4.0 mdurl 0.1.2 mpmath 1.3.0 msvc_runtime 14.40.33807 multidict 6.0.5 mypy-extensions 1.0.0 nerfacc 0.5.3 networkx 3.3 ninja 1.11.1.1 numba 0.60.0 numpy 2.1.2 nvdiffrast 0.3.1 omegaconf 2.3.0 onnxruntime 1.19.2 open_clip_torch 2.26.1 opencv-python 4.10.0.84 opencv-python-headless 4.10.0.84 packaging 24.1 pathspec 0.12.1 peft 0.13.1 pillow 10.4.0 pip 24.2 platformdirs 4.3.6 pluggy 1.5.0 plyfile 1.1 pointnet2_ops 3.0.0 pooch 1.8.2 portalocker 2.10.1 protobuf 5.28.2 psutil 6.0.0 pycparser 2.22 pygit2 1.15.1 PyGithub 2.4.0 pygltflib 1.16.2 Pygments 2.18.0 pyhocon 0.3.61 PyJWT 2.9.0 PyMatting 1.1.12 PyMCubes 0.1.6 pymeshlab 2023.12.post2 PyNaCl 1.5.0 pyparsing 3.1.4 pyreadline3 3.5.4 python-dateutil 2.9.0.post0 pytorch-lightning 2.4.0 pytorch-msssim 1.0.0 pytorch3d 0.7.8 pywin32 307 PyYAML 6.0.2 referencing 0.35.1 regex 2024.7.24 rembg 2.0.59 requests 2.32.3 rich 13.9.2 rpds-py 0.20.0 safetensors 0.4.5 scikit-image 0.24.0 scipy 1.14.1 sentencepiece 0.2.0 setuptools 74.1.2 shellingham 1.5.4 simple_knn 0.0.0 six 1.16.0 slangtorch 1.2.6 smmap 5.0.1 soundfile 0.12.1 spandrel 0.3.4 sympy 1.13.3 tabulate 0.9.0 termcolor 2.5.0 tifffile 2024.9.20 timm 1.0.9 tokenizers 0.19.1 torch 2.4.0+cu121 torch_scatter 2.1.2+pt24cu118 torchaudio 2.4.0+cu121 torchmetrics 1.4.2 torchsde 0.2.6 torchtyping 0.1.5 torchvision 0.19.0+cu121 tqdm 4.66.5 trampoline 0.1.2 transformers 4.44.2 trimesh 4.4.9 trove-classifiers 2024.9.12 typeguard 2.13.3 typer 0.12.5 typing_extensions 4.12.2 typing-inspect 0.9.0 urllib3 1.26.20 wcwidth 0.2.13 wheel 0.44.0 wrapt 1.16.0 xatlas 0.0.9 xformers 0.0.27.post2 yacs 0.1.8 yarl 1.9.11 zipp 3.20.2

Seems Torch libraries no longer show cu118, instead it is cu121 all except the torch_scatter!

But doing all of this brung me a whole new problem:

A module that was compiled using NumPy 1.x cannot be run in NumPy 2.1.2 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2.

What to do now?

YanWenKun commented 1 month ago

@GPU-server

To downgrade Numpy:

python.exe -s -m pip install numpy<2

And I noticed that you're using CUDA 11.8 packages, which should be fine with current setup, but in case you want to downgrade from cu121 to cu118:

python.exe -s -m pip install --force-reinstall xformers==0.0.27.post2 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --extra-index-url https://pypi.org/simple
GPU-server commented 1 month ago

@YanWenKun where did yousee the 11.8 packages, it was only torch_scatter right? Which I upgraded to cu121 ( I posted 2 messages with 2 pip lists, check it) Anyway I think I just made it work, I also had to do something (changing code of a script in the gpytoolbox library" I CANNOT BELIEVE ITS WORKING LOL. Are the workflow examples eady to use? It seems there are many and I dont know what they all do?

GPU-server commented 1 month ago

Never mind it does not work :( I dont get any "failed import" for the extension, but first workflow I tried and I got: RuntimeError: Numpy is not available Eventhough it is on my list pip, look:

Package Version


accelerate 1.0.0 aiohappyeyeballs 2.4.0 aiohttp 3.10.5 aiosignal 1.3.1 antlr4-python3-runtime 4.9.3 attrs 24.2.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 cmake 3.30.4 colorama 0.4.6 coloredlogs 15.0.1 contourpy 1.3.0 cryptography 43.0.1 cycler 0.12.1 dataclasses-json 0.6.7 Deprecated 1.2.14 diff_gaussian_rasterization 0.0.0 diffusers 0.30.3 einops 0.8.0 filelock 3.16.1 flatbuffers 24.3.25 fonttools 4.54.1 frozenlist 1.4.1 fsspec 2024.9.0 ftfy 6.2.3 fvcore 0.1.5.post20221221 gitdb 4.0.11 GitPython 3.1.43 gpytoolbox 0.3.2 hatchling 1.25.0 huggingface-hub 0.24.6 humanfriendly 10.0 idna 3.8 imageio 2.35.1 imageio-ffmpeg 0.5.1 importlib_metadata 8.5.0 iopath 0.1.10 jaxtyping 0.2.34 Jinja2 3.1.4 jsonschema 4.23.0 jsonschema-specifications 2024.10.1 kiui 0.2.13 kiwisolver 1.4.7 kornia 0.7.3 kornia_rs 0.1.5 lazy_loader 0.4 lightning-utilities 0.11.7 llvmlite 0.43.0 markdown-it-py 3.0.0 MarkupSafe 3.0.1 marshmallow 3.22.0 matplotlib 3.9.2 matrix-client 0.4.0 mdurl 0.1.2 mpmath 1.3.0 msvc_runtime 14.40.33807 multidict 6.0.5 mypy-extensions 1.0.0 nerfacc 0.5.3 networkx 3.3 ninja 1.11.1.1 numba 0.60.0 numpy 2.0.0 nvdiffrast 0.3.1 omegaconf 2.3.0 onnxruntime 1.19.2 open_clip_torch 2.26.1 opencv-python 4.10.0.84 opencv-python-headless 4.10.0.84 packaging 24.1 pathspec 0.12.1 peft 0.13.1 pillow 10.4.0 pip 24.2 platformdirs 4.3.6 pluggy 1.5.0 plyfile 1.1 pointnet2_ops 3.0.0 pooch 1.8.2 portalocker 2.10.1 protobuf 5.28.2 psutil 6.0.0 pycparser 2.22 pygit2 1.15.1 PyGithub 2.4.0 pygltflib 1.16.2 Pygments 2.18.0 pyhocon 0.3.61 PyJWT 2.9.0 PyMatting 1.1.12 PyMCubes 0.1.6 pymeshlab 2023.12.post2 PyNaCl 1.5.0 pyparsing 3.1.4 pyreadline3 3.5.4 python-dateutil 2.9.0.post0 pytorch-lightning 2.4.0 pytorch-msssim 1.0.0 pytorch3d 0.7.8 pywin32 307 PyYAML 6.0.2 referencing 0.35.1 regex 2024.7.24 rembg 2.0.59 requests 2.32.3 rich 13.9.2 rpds-py 0.20.0 safetensors 0.4.5 scikit-image 0.24.0 scipy 1.14.1 sentencepiece 0.2.0 setuptools 74.1.2 shellingham 1.5.4 simple_knn 0.0.0 six 1.16.0 slangtorch 1.2.6 smmap 5.0.1 soundfile 0.12.1 spandrel 0.3.4 sympy 1.13.3 tabulate 0.9.0 termcolor 2.5.0 tifffile 2024.9.20 timm 1.0.9 tokenizers 0.19.1 torch 2.4.0+cu121 torch_scatter 2.1.2+pt24cu121 torchaudio 2.4.0+cu121 torchmetrics 1.4.2 torchsde 0.2.6 torchtyping 0.1.5 torchvision 0.19.0+cu121 tqdm 4.66.5 trampoline 0.1.2 transformers 4.44.2 trimesh 4.4.9 trove-classifiers 2024.9.12 typeguard 2.13.3 typer 0.12.5 typing_extensions 4.12.2 typing-inspect 0.9.0 urllib3 1.26.20 wcwidth 0.2.13 wheel 0.44.0 wrapt 1.16.0 xatlas 0.0.9 xformers 0.0.27.post2 yacs 0.1.8 yarl 1.9.11 zipp 3.20.2

What do you think?

YanWenKun commented 1 month ago

@GPU-server

python.exe -s -m pip install --force-reinstall numpy==1.26.4
GPU-server commented 1 month ago

Thanks. How did you know that was the exact version we needed? Btw, did you get the problem of having your torch being unisntalled each time you isntall missing nodes? (https://github.com/MrForExample/ComfyUI-3D-Pack/issues/341)

YanWenKun commented 1 month ago

@GPU-server

Thanks. How did you know that was the exact version we needed?

Just use PyPI to find the package you need: https://pypi.org/project/numpy/#history

did you get the problem of having your torch being unisntalled each time you isntall missing nodes?

Just seperate Python environments. One for Comfy3D, one for everything else, etc. In portable packages, each python_embeded manage its own pip packages and isolate from the other.

I use GitHub to build big all-in-one package. It covers most popular nodes. And I never update pip packages in one environment, I just replace the whole Python environment, like swap-in and swap-out CDs. It's faster, easier, and dependency-resolved. Also I rarely use ComfyUI-Manger to install/update custom nodes. Using git is much faster, and it won't try to fix Python dependencies like Manager did. But it won't install missing packages, too.

Btw, I'm curious which node is asking for updating PyTorch from 2.4.0 to 2.4.1 ?

YanWenKun commented 1 month ago

p.s. Edit extra_model_paths.yaml in ComfyUI folder to share one set of models across multiple ComfyUI instances.

GPU-server commented 1 month ago

Hi again @YanWenKun, Did you mean one venv for torch3d? and the rest of components from comfy 3D pack extension in another venv? Or did you simply mean one single venv for the WHOLE comfy 3D pack? And another venv for ALL other extensions?


Almost all extensions (not all and I am afraid to experiment) uninstall Torch 2.4.0 and replace it with 2.4.1 (and forget to "inform" comfyui it did, so it can not even start?), it also UPDATE Numpy to the latest version. Which is a problme (as you saw earlier)

One in mind is probably Comfy Essentials. (which did not even install actually, after breaking my comfy)


Here is my method of installation: Download ComfyUI portable 2.4.1 Download Comfy manager (and leave it) Git clone Comfy 3D pack Make sure cuda tooklit 12.1 is installed Uninstall Torch 2.4.1 install Torch 2.4.0 cu121 Run python.exe install.py Try to resolve every error: For example missing torch_scatter, installing NINJA libary, fixing error related gpytoolbox (changing code of reach for the arcs . py, changing np.Inf to np.inf with an minor i), downgrading numpy (as you saw)..

After that miraculously: few workflow work:

Here is where I am actually. Can you describe your method step by step?

YanWenKun commented 1 month ago

@GPU-server

  1. Download this for non-3D-Pack usage.
  2. Download this for 3D-pack usage. Delete folders you don't need in custom_nodes.
  3. (Optional) Under ComfyUI folder, rename extra_model_paths.yaml.example to extra_model_paths.yaml. Edit it so two ComfyUIs can share one models folder. You can skip this step if one ComfyUI is dedicated to 3D-Pack, as 3D-Pack will download models to its own folder under custom_nodes.

And lastly, you can fork this repo, add/remove nodes in stage2.sh, then build it using GitHub Actions, so you'll have a ComfyUI portable of your own.

GPU-server commented 1 month ago

Thanks @YanWenKun But I wanted to learn the hard version, the version where we install it from base comfyui Official repo. Do you remember how you did it before making that repo that is ready to use? I want the hard way (to become an expert (like you lol)). ?

GPU-server commented 1 month ago

@GPU-server

Thanks. How did you know that was the exact version we needed?

Just use PyPI to find the package you need: https://pypi.org/project/numpy/#history

Ok but how did you know by visiting that website (https://pypi.org/project/numpy/1.26.4/) that this is the right version for us I mean?

YanWenKun commented 1 month ago

@GPU-server Wow, that's cool!

Tbh, I didn't choose the hard way in this case. I analyzed the build scripts of ComfyUI official portable and rewrote them into scripts of my own flavor.

But if you want to build a ComfyUI bundle from the ground up, the key steps are:

  1. Setup OS environment: Git, GPU drivers (or CUDA Toolkit), C++ build tools.
  2. Setup Python venv, or embeded. Then setup PIP.
  3. Install PyTorch + xFormers. Each version of xFormers is paired with specific PyTorch version, so you could choose the PyTorch version by choosing xFormers' version.
  4. Git clone ComfyUI and custom nodes.
  5. Install requirements.txt of ComfyUI. When done, a basic envrionment for running SD image generation is ready.
  6. Install requirements.txt from custom nodes. Which is overwhelmingly full of fine details. And you should look into them, one by one.
  1. Comfy3D's dependency is complicated and needs special attention. You can use its installer, but manual installation gives you more control. You should examine the scripts and pick the proper wheels to install, and copy the includes and libs for later C++ compiling.

  2. Finally, put models in and let Python run the main.py.

YanWenKun commented 1 month ago

Ok but how did you know by visiting that website (https://pypi.org/project/numpy/1.26.4/) that this is the right version for us I mean?

Numpy2 brings break changes and many projects can't simply upgrade version number from 1.x to 2.x. 1.26.4 is the last (maybe final) working version of Numpy1. PyPI is the repository of PIP.

GPU-server commented 1 month ago

@GPU-server Wow, that's cool!

Tbh, I didn't choose the hard way in this case. I analyzed the build scripts of ComfyUI official portable and rewrote them into scripts of my own flavor.

But if you want to build a ComfyUI bundle from the ground up, the key steps are:

  1. Setup OS environment: Git, GPU drivers (or CUDA Toolkit), C++ build tools.
  2. Setup Python venv, or embeded. Then setup PIP.
  3. Install PyTorch + xFormers. Each version of xFormers is paired with specific PyTorch version, so you could choose the PyTorch version by choosing xFormers' version.
  4. Git clone ComfyUI and custom nodes.
  5. Install requirements.txt of ComfyUI. When done, a basic envrionment for running SD image generation is ready.
  6. Install requirements.txt from custom nodes. Which is overwhelmingly full of fine details. And you should look into them, one by one.
  • onnxruntime-gpu needs a specific repo to download wheels from.
  • cupy-cuda12x needs manual install.
  • ninja needs binary from official, or just OS install configured with PATH.
  • dlib and insightface needs to compile on site, or download binary wheel files from 3rd party.
  • CLIP needs to install from source.
  • gradio is incompatible with ComfyUI-Manager as their dependency on urllib3 is different.
  1. Comfy3D's dependency is complicated and needs special attention. You can use its installer, but manual installation gives you more control. You should examine the scripts and pick the proper wheels to install, and copy the includes and libs for later C++ compiling.
  2. Finally, put models in and let Python run the main.py.

I actually woudl have liked if you described the part number 7 more, but its ok for now. I will try it another day and see if I can. (I was able to make some workflow work Happy withthat for now. It was difficult overall)