search

MrForExample / ComfyUI-3D-Pack

An extensive node suite that enables ComfyUI to process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc.)
MIT License
2.16k stars 212 forks source link

ExplicitTarget Mesh Optimization: allow_pickle=False,lambdalabs/sd-image-variation-diffusers,diffusion_pytorch_model.safetensors #197

Open Titances opened 2 months ago

Titances commented 2 months ago

Ask for help, when I run ExplicitTarget Mesh Optimization, the following error occurs, the online information says that allow_pickle=True is added when compiling, numpy.load is set to False by default after 1.16.3, I want to reduce the version, But there seems to be a mistake, I now have numpy version 1.26.4. The program runs automatically, and I don't know where to add it, or how to keep the pickle open.Or I'm not asking the right question?

1720625938780 1720625993717
Titances commented 2 months ago

I have downloaded the Unique3D model

1720627416425

Run the workflow, but to the ExplicitTarget Mesh Optimization node, it will be paused and lose connection

1720627457168
MrForExample commented 2 months ago

Hi, I'm not sure why can't it download the model automatically But if you download the model manually then Wuvin\Unique3D should be put under Checkpoints\Diffusers folder

Let me know if there are any other issues, Cheers

Titances commented 2 months ago

Hello, sorry to bother you, I updated your latest warehouse, but this problem is still found at runtime image While in the load diffusers pipeline, there is no image2normal, I re-downloaded the entire file and put it in image I chose to skip the duplicate file and didn't overwrite it, but it still indicated that the model was missing,I didn't find this location in my folder.But I found the address of the hugging face file image https://huggingface.co/lambdalabs/sd-image-variations-diffusers/tree/main Do I need to download them all, and where do I need to put the files to identify them,Thank you for your answer

Titances commented 2 months ago

Is this an automatic download? is the file suffix not matching, one is.safetensors, the other is.bin, or is this node calling files with the same name in other locations? image image Looking forward to your reply

zafirexd commented 2 months ago

@Titances Same error, maybe if try to put the whole "sd-image-blahblahblah" folder inside diffusers folder, image

Titances commented 2 months ago

@zafirexd Hi, I have placed the sd-image-variables-diffusers folder in the following locations ComfyUI-3D-Pack ComfyUI-3D-Pack\Checkpoints ComfyUI-3D-Pack\Checkpoints\Diffusers ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D\image2normal ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D\image2normal\vae ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D\image2normal\image_encoder ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D\image2mvimage ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D\image2mvimage\vae ComfyUI-3D-Pack\Checkpoints\Diffusers\Wuvin\Unique3D\image2mvimage\image_encoder But the same problem still arises image How did you solve it

MrForExample commented 2 months ago

@Titances As for lambdalabs--sd-image-variations-diffusers, model should be downloaded automatically at your cache location, e.g. for me is at: C:\Users\reall.cache\huggingface\hub\models--lambdalabs--sd-image-variations-diffusers I'm not sure exactly what is the issue in your case, I have never seen an error like this, but it seems error inside the huggingface_hub.snapshot_download function. Let me take a look and get back to you.

@zafirexd It seems there is a git merge error there as there shouldn't be a checkpoints folder, but instead should be Checkpoints Captial letter, I just fixed that, please give a try again๐Ÿ‘

MrForExample commented 2 months ago

@Titances Hi, I updated Diffusers Download Node & Unique3D related workflows, feel free to give a try and let me know if you encountered other problem

Titances commented 2 months ago

image image Hi, I created a new ComfyUI with only manager and 3D package. People who know a little about programming tell me that the program (model_index.Json) cannot be found during command execution. Put (model index.Json) in 'E:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-3D-Pack\Checkpoints\Diffusers\MrForExample\Unique3D' The system prompts that the (scheduler config.json) file is not found What should I do? ComfyUI_3D_Pack is fully installed

zafirexd commented 2 months ago

After force download on all 2 times, error still occurs same as this:

image

When the process reach the 0/100 steps after the 200/200 steps like this, comfy just die at this point: image

in CharacterGen_to_Unique3D workflow, @MrForExample can you show how the models in "C:\Users\reall.cache\huggingface\hub\models--lambdalabs--sd-image-variations-diffusers" looks like?, i mean, if are 2 diffusion_pytorch_model.safetensors inside and where to download them, because in code the format is .bin, image or if are the same as this models: https://huggingface.co/lambdalabs/sd-image-variations-diffusers/tree/main

@Titances try this dedicate comfy instance: https://github.com/YanWenKun/ComfyUI-Windows-Portable/releases/tag/v3.11

CharacterGen_Vanilla works well. image

CharacterGen_InstantMesh.....ah.... here we go again...... image

i tried to make a hard reset to previous version and try some things, i find something with the config.yaml files and this image I see something needs this "no roma module" and more things, i update and tested from this point, image and all works well, then i update to the charagen update and many things were broke, and i see a lot of changes in the config.yaml files, i think the paths does not match and other problems, im not 100% sure, im gonna try to replicate my old environment for the last commit all works well for me.

MrForExample commented 2 months ago

Hi @zafirexd I'm not sure what exactly cased the issue in ExplicitTarget Mesh Optimization node, but this node only runs optimization algorithm directly on a 3d mesh which doesn't use any pre-trained model, so I don't think lambdalabs--sd-image-variations-diffusers caused the issue.

And please use the newest version as I fixed bunch of issue yesterday, folder name is one of it, and please post any error you encountered here I'll see what I can help with, cheers ๐Ÿ‘

zafirexd commented 2 months ago

@MrForExample i dont know, maybe pytorch/xformers, or the specific requirements per version, image (update) image(old f3cbd66 commit)

image all its working perfectly in f3cbd66 commit https://github.com/MrForExample/ComfyUI-3D-Pack/tree/f3cbd66020d109cbfd4c708e9d989245af752768 i downloaded as zip and extract as manual custom node install in this comfy instance https://github.com/YanWenKun/ComfyUI-Windows-Portable/releases/tag/v3.9 Anyway, CharaGen is working well on update, so i start to manage the both instances, I have win10 and rtx 3060 12Gb. So for charagen cut the process and save all the images, image Then load them in with examples/unique 3d/3_Mesh_Reconstruction.json workflow in the commit instance image well......its working xD!, CharaGen its better for anime characters. @MrForExample thanks for the support.

clemecleme commented 1 month ago

Hey there, I have the same problem of ComfyUI disconnecting at the same moment when going through ExplicitTarget Mesh Optimization node, but without any error message. The error I get each time in the Event Viewer is the following, related to matplotlib. I've tried to upgrade / uninstall / reinstall matplotlib but doesn't work. Any idea or similar problem? Thanks a lot!

ComfyUI_crash Eventviewer_error

Titances commented 1 month ago
ๅพฎไฟกๅ›พ็‰‡_20240720140100 ๅพฎไฟกๅ›พ็‰‡_20240720141231

Warn!: H:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\Unique3D\mesh_reconstruction\remesh.py:353: UserWarning: Using torch.cross without specifying the dim arg is deprecated. Please either pass the dim explicitly or simply use torch.linalg.cross. The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ..\aten\src\ATen\native\Cross.cpp:66.) n = torch.cross(e1,cl) + torch.cross(cr,e1) #sum of old normal vectors

Same problem,Hardware? system?I don't know what the problem is.

WorkSync commented 1 month ago

ๅพฎไฟกๅ›พ็‰‡_20240720140100 ๅพฎไฟกๅ›พ็‰‡_20240720141231 Warn!: H:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\Unique3D\mesh_reconstruction\remesh.py:353: UserWarning: Using torch.cross without specifying the dim arg is deprecated.่ญฆๅ‘Š๏ผ๏ผš H:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\Unique3D\mesh_reconstruction\remesh.py:353๏ผšUserWarning๏ผšไธๆŽจ่ไฝฟ็”จ torch.cross ่€ŒไธๆŒ‡ๅฎšๆš—ๆทกๅ‚ๆ•ฐใ€‚ Please either pass the dim explicitly or simply use torch.linalg.cross.่ฏทๆ˜พๅผไผ ้€’ๆš—ๆทกๆˆ–ไป…ไฝฟ็”จ torch.linalg.crossใ€‚ The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ..\aten\src\ATen\native\Cross.cpp:66.) ๅœจๆœชๆฅ็‰ˆๆœฌไธญ๏ผŒdim ็š„้ป˜่ฎคๅ€ผๅฐ†ๆ›ดๆ”นไธบไธŽ linalg.cross ็š„้ป˜่ฎคๅ€ผไธ€่‡ดใ€‚ ๏ผˆๅœจ ..\aten\src\ATen\native\Cross.cpp:66 ๅ†…้ƒจ่งฆๅ‘ใ€‚๏ผ‰ n = torch.cross(e1,cl) + torch.cross(cr,e1) #sum of old normal vectors n = torch.cross(e1,cl) + torch.cross(cr,e1) #ๆ—งๆณ•ๅ‘้‡ไน‹ๅ’Œ

Same problem,Hardware? system?I don't know what the problem is.ๅŒๆ ท็š„้—ฎ้ข˜๏ผŒ็กฌไปถ๏ผŸ็ณป็ปŸ๏ผŸไธ็Ÿฅ้“ๆ˜ฏไป€ไนˆ้—ฎ้ข˜ใ€‚

I meet this problem too.Could anybody help me please...

al3dv2 commented 1 month ago

Same problem its disconnect at ExplicitTarget Mesh Optimization

krizalid38 commented 1 month ago

Same problem for me! disconnect at ExplicitTarget Mesh Optimization

krizalid38 commented 1 month ago

stucks in this stage.

image

100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 30/30 [00:11<00:00, 2.57it/s] [Comfy3D] [Save_3D_Mesh] Saving model to D:\aimodel\ComfyUI_windows_portable\ComfyUI\output\CharacterGen_Unique3D\Mesh_Gen_0\mesh.obj [Comfy3D] [Server Query view_file] Get file D:\aimodel\ComfyUI_windows_portable\ComfyUI\output\CharacterGen_Unique3D\Mesh_Gen_0/mesh.mtl [Comfy3D] [Server Query view_file] Get file D:\aimodel\ComfyUI_windows_portable\ComfyUI\output\CharacterGen_Unique3D\Mesh_Gen_0\mesh.obj 0%| | 0/200 [00:00<?, ?it/s]Warn!: D:\aimodel\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-3D-Pack\Gen_3D_Modules\Unique3D\mesh_reconstruction\remesh.py:353: UserWarning: Using torch.cross without specifying the dim arg is deprecated. Please either pass the dim explicitly or simply use torch.linalg.cross. The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ..\aten\src\ATen\native\Cross.cpp:66.) n = torch.cross(e1,cl) + torch.cross(cr,e1) #sum of old normal vectors

100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 200/200 [00:07<00:00, 25.85it/s] 0%| | 0/100 [00:00<?, ?it/s]

al3dv2 commented 1 month ago

Please help!

xintoc commented 1 month ago

Sadly ExplicitTarget Mesh Optimization causes a crash, which results in a 300MB dmp file in %APPDATA% \Local\CrashDumps\ Analyzing with WinDbg and EventViewer > Application >Error (python_embeded\Lib\site-packages\matplotlib.libs\msvcp140-6547a4e8b7053d61be063c313b1addd0.dll)

*** WARNING: Unable to verify checksum for _C.cp311-win_amd64.pyd

Key Points:

  1. Exception Type:

    • Access Violation (c0000005): The application attempted to read from an invalid memory address ffffffffffffffff.

  2. Faulting Module:

    • msvcp140-6547a4e8b7053d61be063c313b1addd0.dll: This is a specific version of the MSVC++ runtime library.
    • Function: __ExceptionPtr::_RethrowException+0x10c

  3. Process:

    • Name: python.exe
    • Version: 3.11.9150.1013

  4. Call Stack:

    • The stack trace shows a sequence of function calls leading to the access violation.
    • Key functions involved:
      • msvcp140_6547a4e8b7053d61be063c313b1addd0!__ExceptionPtr::_RethrowException
      • _image_cp311_win_amd64
      • _C_cp311_win_amd64
      • VCRUNTIME140_1!CallSettingFrame_LookupContinuationIndex
      • python311!PyObject_Call
      • python311!PyEval_EvalFrameDefault
      • torch_python!THPPointer<THPStorage>::THPPointer<THPStorage>

Analysis:

  1. Invalid Memory Access:

    • The access violation occurred because the code attempted to read from an invalid memory address (ffffffffffffffff), which is typically indicative of a null pointer dereference or an invalid pointer access.

  2. Involvement of MSVC++ Runtime:

    • The exception is rethrown using MSVC++ runtime libraries, suggesting an issue in handling an exception in the C++ code that interacts with the Python module.

  3. _C.cp311-win_amd64.pyd Module:

    • This appears to be a compiled Python extension module, likely a C++ or C module used in the Python environment. The problem might be originating from this module or its interaction with other libraries or Python itself.

So far indeed only this version combination seems to work: https://github.com/MrForExample/ComfyUI-3D-Pack/tree/f3cbd66020d109cbfd4c708e9d989245af752768 https://github.com/YanWenKun/ComfyUI-Windows-Portable/releases/tag/v3.9 These work with CUDA 12.4.1 and Visual Studio Build Tools 17.10.5 installed, so that is not the conflict.

Anything newer than these two ComfyUI and 3D-Pack combos fail at "ExplicitTarget Mesh Optimization".