search

MrForExample / ComfyUI-3D-Pack

An extensive node suite that enables ComfyUI to process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc.)
MIT License
2.17k stars 214 forks source link

Pack working for me after the following changes #300

Open drake7707 opened 1 week ago

drake7707 commented 1 week ago

I had a lot of issues getting this pack to work in linux in a docker container,

however after many hours of slow progress I finally got it to work. Here are the changes I needed to make:

1) Don't use prebuilt wheels, it gives errors, I had to build them manually, I'm using "pytorch/pytorch:2.4.0-cuda12.1-cudnn9-devel" as base image

2) Do not use autobuild, it will pull the wrong dependencies for pytorch 2.4.0, I installed torch-scatter with pip install torch-scatter==2.1.2 -f https://data.pyg.org/whl/torch-2.4.0+cu121.html and xformers==0.0.27.post2 through pip and torch-vision was already installed

3) I built the diff-gaussian-rasterization, gaussian-splatting -> submodules/simple-knn (had to git clone this one), kiuikit, nvdiffrast and pytorch3d all through their respective python setup.py install.

4) Don't use OpenGL, I could not get it to work and kept getting eglInitialize() failed errors. So I had to replace all RasterizeGLContext with RasterizeCudaContext

5) I had to change the requirements: needed to make sure of pygltflib==1.16.2 and had to add dataclasses-json

At this point the container started with the 3d pack loaded succesfully

Buuuuut I still had issues in the frontend as well, even on the latest ComfyUI version:

6) The threeVisualiser preview loads in an iframe. For reasons I don't fully understand it tries to import the comfyUI api from window.comfyUI, but this window context is the child window, not the parent window. So I had to patch the comfyUI scripts/api.js shim with export const api = window.comfyAPI?.api?.api || parent.window.comfyAPI?.api?.api;

7) That loaded the node preview without exceptions but threeVisualiser.js uses api.apiURL, which refers to the /api/viewfile and not to the extension path, so I had to patch that:

-        currentURL = api.apiURL('/viewfile?' + new URLSearchParams(params));
+        currentURL = api.fileURL('/extensions/ComfyUI-3D-Pack/html/viewfile?' + new URLSearchParams(params));

And

-            mtlLoader.setPath(api.apiURL('/viewfile?' + new URLSearchParams({"filepath": mtlFolderpath})));
+            mtlLoader.setPath(api.apiURL('/extensions/ComfyUI-3D-Pack/html/viewfile?' + new URLSearchParams({"filepath": mtlFolderpath})));

I don't really know what went wrong in the frontend there, but that fixed it for me, I now have a fully working workflow and a preview

Not sure if it will help anyone but I've pushed my image, run it with:

docker run -d \
--name comfyui_3d \
-p 8188:8188 \
--gpus=all \
--restart unless-stopped \
-v $(pwd)/data/input:/comfyui/input \
-v $(pwd)/data/models:/comfyui/models \
-v $(pwd)/data/output:/comfyui/output \
-v $(pwd)/data/custom_nodes:/comfyui/custom_nodes \
-v $(pwd)/data/my_workflows:/comfyui/my_workflows \
drake7707/comfyui:3d

make sure your data folder exist for the bind mount:

mkdir -p data/input
mkdir -p data/output
mkdir -p data/models
mkdir -p data/custom_nodes
mkdir -p data/my_workflows

I've commited my changes on my fork: https://github.com/drake7707/ComfyUI-3D-Pack/commit/4aa5e86218b10403a7fe9de47cbdfa0731192fbc

So if you pull https://github.com/drake7707/ComfyUI-3D-Pack.git into your data/custom_nodes folder, run pip install -r requirements in the container and restart, it should all work

drake7707 commented 3 days ago

torch-scatter needs to be installed with pip install torch-scatter==2.1.2 -f https://data.pyg.org/whl/torch-2.4.0+cu121.html or it's not pulling the CUDA support.

Also needed diffusers==0.29.0 to prevent errors in _get_model_file