search

ShmuelRonen / ComfyUI_wav2lip

A custom node for ComfyUI that allows you to perform lip-syncing on videos using the Wav2Lip model. It takes an input video and an audio file and generates a lip-synced output video.
52 stars 9 forks source link

Hi there is a problem Cloning the repository (file not found) and also loading the wav2lip_gan.pth from the folder (C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_wav2lip\Wav2Lip\checkpoints) it only fined the models in the folder (C:\ComfyUI_windows_portable\ComfyUI\models\facerestore_models) #30

Open gilushon opened 1 week ago

gilushon commented 1 week ago

היי שמואל, תודה רבה על ה-NODE הנפלא, יש בעיה לתפעל אותו .....

Hi there is a problem Cloning the repository (file not found) and also loading the wav2lip_gan.pth from the folder (C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_wav2lip\Wav2Lip\checkpoints) it only fined the models in the folder (C:\ComfyUI_windows_portable\ComfyUI\models\facerestore_models)

The only model that working is codeformer.

The main error after doing all the installation steps is:

Error occurred when executing FaceRestoreModelLoader:

File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\facerestore_cf__init.py", line 293, in load_model out = model_loading.load_state_dict(sd).eval() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable\ComfyUI\comfy_extras\chainner_models\model_loading.py", line 5, in load_state_dict return ModelLoader().load_from_state_dict(state_dict).eval() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\spandrel__helpers\loader.py", line 94, in load_from_state_dict return self.registry.load(state_dict).to(self.device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\spandrel\helpers\registry.py", line 184, in load raise UnsupportedModelError

Close Queue size: 0 ⚙️ 🖼️ × Queue Prompt Extra options Queue FrontView QueueView History Save LoadRefresh Clipspace Clear Load Default Reset View Manager Share spandrel.__helpers.registry.UnsupportedModelError 142 FaceRestoreModelLoader

מחכה לעזרה, או תיקון, תודה רבה !!!

ShmuelRonen commented 1 week ago

Hi Gil. I also have the problem than in facerestore_cf node only codeformer works. Unfortunately the facerestore_cf node has not been updated for about 9 months and maybe because of that it probably having trouble functioning.

gilushon commented 1 week ago

Hi, so the only option is to use only the codeformer model? (it's not that good model...) will you update your wa2lip project to work with all the models?

ShmuelRonen commented 1 week ago

I will try to figure it out.

gilushon commented 4 days ago

your solution to lip-sync in ComphyUi is very good, I am testing for the last 5 days other solutions, or they are not working, or they are giving shitty results, CodeFormer on 0.5 and upscaling with Real ESRGAN 4x give decent results, but you need to do masking of the mouth because the face is shaking and also the mouth a bit. I also tried to install your Video-Retaking project and it doesn't work :( I hope to find a good solution because in the meantime you are my only hope :)

Kimthanh27 commented 1 day ago

Error occurred when executing FaceRestoreCFWithModel:

Unable to allocate 31.6 MiB for an array with shape (3840, 2160, 1) and data type int32

File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) Every time I use the face enhance flow, there is an issue with face restore. Have you found a solution yet?