MrForExample / ComfyUI-3D-Pack

An extensive node suite that enables ComfyUI to process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc.)
MIT License
1.73k stars 167 forks source link

TripoSR bad geometry generation #118

Closed kakachiex2 closed 3 months ago

kakachiex2 commented 3 months ago

Suddenly my workflow shows this mesh; I create a setup with the Flowty TripoSR to see if it do the same but work ok. Screenshot 2024-03-19 090420

kakachiex2 commented 3 months ago

I have small feature request I like to resize the 3d viewer to match my workflow proportion and keep good organization but wen loading the workflow the viewer go back to its default size. Another feature is to connect the viewer directly from the Switch Mesh Axis not always want to save the mesh.

MrForExample commented 3 months ago

Suddenly my workflow shows this mesh; I create a setup with the Flowty TripoSR to see if it do the same but work ok. Screenshot 2024-03-19 090420

Yeah, about this problem, you probably omitted invert mask node somewhere, since by default the mask as a image is inverted when you load the image (By default object part of mask is black and background is white, and you want the opposite before feeding it into CRM's nodes)

MrForExample commented 3 months ago

I have small feature request I like to resize the 3d viewer to match my workflow proportion and keep good organization but wen loading the workflow the viewer go back to its default size. Another feature is to connect the viewer directly from the Switch Mesh Axis not always want to save the mesh.

Sure, sounds reasonable enough, I'll see what I can do 👍

kakachiex2 commented 3 months ago

It has the invert mask if you see the screen, it says get invert mask, I tested with two mode normal mask and invert mask and keep doing the same.

kakachiex2 commented 3 months ago

@MrForExample I create a new TripoSR workflow from start with the same configuration and my first TripoSR workflow old one.

Old Workflow :: Screenshot 2024-03-22 140131

New Workflow :: Screenshot 2024-03-22 140151

kakachiex2 commented 3 months ago

I create a new test workflow for TripoSR and know I'm getting this new errors.

Error occurred when executing [Comfy3D] TripoSR:

Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)

File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(*slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI\ComfyUI_Ex\python_miniconda_env\ComfyUI\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI\ComfyUI_Ex\ComfyUI\custom_nodes\ComfyUI-3D-Pack\nodes.py", line 1653, in run_TSR image = torch.cat((image, mask), dim=2).detach().cpu().numpy() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

kakachiex2 commented 3 months ago

Ok I found the reason I was using BIREfNet to extract the mask and after processing the invert mask to TripoSR it gives these errors but know I'm using BRIA Remove background for the mask and don't throw these errors but keep generating bad mesh.

Screenshot 2024-03-22 151929

kakachiex2 commented 3 months ago

After several test looks like something change and your TripSR is not taking into account the inverted mask it only works with normal mask and the mesh looks les detailed. Here Im using not inverted mask and it works :

Screenshot 2024-03-22 163149