search

SamKhoze / ComfyUI-DeepFuze

DeepFuze is a state-of-the-art deep learning tool that seamlessly integrates with ComfyUI to revolutionize facial transformations, lipsyncing, Face Swapping, Lipsync Translation, video generation, and voice cloning.
https://cogidigm.com/
Other
213 stars 13 forks source link

Switched to VideoHelper Load Audio (Error) Missing a file?? #8

Open rethink-studios opened 1 week ago

rethink-studios commented 1 week ago

got prompt '🔥 - 11 Nodes not included in prompt but is activated' codeformer 24.0 video/h265-mp4 video h265-mp4 {'enhancer': 'codeformer', 'frame_enhancer': 'real_esrgan_x2', 'face_mask_padding_left': 0, 'face_mask_padding_right': 0, 'face_mask_padding_bottom': 0, 'face_mask_padding_top': 0, 'trim_frame_start': 0, 'trim_frame_end': 0, 'device': 'cpu', 'frame_rate': 24.0, 'loop_count': 0, 'filename_prefix': 'deepfuze', 'pingpong': False, 'save_output': True, 'images': ['2', 0], 'audio': ['22', 0], 'format': 'video/h265-mp4', 'pix_fmt': 'yuv420p10le', 'crf': 22, 'save_metadata': ['save_metadata', 'BOOLEAN', {'default': True}]} D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite\videohelpersuite..\video_formats\h265-mp4.json pix_fmt crf save_metadata Output images were not of valid resolution and have had padding applied D:\COMFYUI_BETA\ComfyUI\input\audio\17188337500601833.wav ['python', './run.py', '--frame-processors', 'lip_syncer', '-s', 'D:\COMFYUI_BETA\ComfyUI\input\audio\17188337500601833.wav', '-t', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze_00011.mp4', '-o', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\1718833750069751.mp4', '--trim-frame-start', '0', '--trim-frame-end', '310', '--face-mask-padding', '0', '0', '0', '0', '--headless'] Traceback (most recent call last): File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\run.py", line 7, in from deepfuze import core File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\core.py", line 16, in from deepfuze.face_analyser import get_one_face, get_average_face File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_analyser.py", line 10, in from deepfuze.face_helper import estimate_matrix_by_face_landmark_5, warp_face_by_face_landmark_5, warp_face_by_translation, create_static_anchors, distance_to_face_landmark_5, distance_to_bounding_box, convert_face_landmark_68_to_5, apply_nms, categorize_age, categorize_gender File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_helper.py", line 2, in from cv2.typing import Size ModuleNotFoundError: No module named 'cv2.typing' ['python', './run.py', '--frame-processors', 'face_enhancer', '--face-enhancer-model', 'codeformer', '-t', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\1718833750069751.mp4', '-o', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4', '--headless'] Traceback (most recent call last): File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\run.py", line 7, in from deepfuze import core File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\core.py", line 16, in from deepfuze.face_analyser import get_one_face, get_average_face File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_analyser.py", line 10, in from deepfuze.face_helper import estimate_matrix_by_face_landmark_5, warp_face_by_face_landmark_5, warp_face_by_translation, create_static_anchors, distance_to_face_landmark_5, distance_to_bounding_box, convert_face_landmark_68_to_5, apply_nms, categorize_age, categorize_gender File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_helper.py", line 2, in from cv2.typing import Size ModuleNotFoundError: No module named 'cv2.typing' ['python', './run.py', '--frame-processors', 'frame_enhancer', '--frame-enhancer-model', 'real_esrgan_x2', '-t', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4', '-o', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4', '--headless'] Traceback (most recent call last): File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\run.py", line 7, in from deepfuze import core File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\core.py", line 16, in from deepfuze.face_analyser import get_one_face, get_average_face File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_analyser.py", line 10, in from deepfuze.face_helper import estimate_matrix_by_face_landmark_5, warp_face_by_face_landmark_5, warp_face_by_translation, create_static_anchors, distance_to_face_landmark_5, distance_to_bounding_box, convert_face_landmark_68_to_5, apply_nms, categorize_age, categorize_gender File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_helper.py", line 2, in from cv2.typing import Size ModuleNotFoundError: No module named 'cv2.typing' ffmpeg version 2024-02-15-git-a2cfd6062c-essentials_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband libavutil 58. 38.100 / 58. 38.100 libavcodec 60. 39.100 / 60. 39.100 libavformat 60. 21.100 / 60. 21.100 libavdevice 60. 4.100 / 60. 4.100 libavfilter 9. 17.100 / 9. 17.100 libswscale 7. 6.100 / 7. 6.100 libswresample 4. 13.100 / 4. 13.100 libpostproc 57. 4.100 / 57. 4.100 [in#0 @ 000001e0158e0e40] Error opening input: No such file or directory Error opening input file D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4. Error opening input files: No such file or directory None None !!! Exception during processing!!! _D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4 could not be loaded with cv. Traceback (most recent call last): File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\nodes.py", line 1425, in lipsyncgenerate return load_video_cv(filename,0,'Disabled',512,512,0,0,1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\nodes.py", line 543, in load_video_cv (width, height, fps, duration, total_frames, target_frame_time) = next(gen) ^^^^^^^^^ File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\nodes.py", line 462, in cv_frame_generator raise ValueError(f"{video} could not be loaded with cv.") ValueError: _D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4 could not be loaded with cv.

Prompt executed in 4.59 seconds

image

SamKhoze commented 1 week ago

Please make sure to read the instructions before seeking assistance. VHS node is not compatible to load audio for deepfuze We suggeste.

rethink-studios commented 1 week ago

I understand, and trust me, I've tried both nodes. :) Neither works, and I get the same error. As you can see in the image, the other node is in the canvas

SamKhoze commented 1 week ago

@rethink-studios If you try both perhaps it's not installed correctly, I am sorry if this gives you a hard time, I wish I could find a one-click solution like other nodes, Lipsync, Faceswapping, Voice cloning, each of them are very complex even outside of comfyui it requires understanding the errors and know how to fix it. Let's try one more time, restart your comfyui and go to your terminal window, if you see any place "Error" or "NoModuleFound" take a screenshot and share it here.

rethink-studios commented 1 week ago

WE BELIEVE IN YOU!! :)

attached is the error log in Comfy..

searching for "error" if meet No module named 'hallo' error,please restart comfyui

ImportError: cannot import name 'create_vae_diffusers_config' from 'diffusers.loaders.single_file_utils' (D:\COMFYUI_BETA\python_embeded\Lib\site-packages\diffusers\loaders\single_file_utils.py)

raise ImportError("Diffusers version too old. Please update to 0.26.0 minimum.")

ImportError: Diffusers version too old. Please update to 0.26.0 minimum.

and this:

No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'

deepFUZE_error2.txt

rethink-studios commented 1 week ago

When I run this: image

I get this:

To see the GUI go to: http://127.0.0.1:8188 FETCH DATA from: D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE] BilboX LUTs path set to: D:\COMFYUI_BETA\ComfyUI\custom_nodes\bilbox-comfyui\luts got prompt '🔥 - 4 Nodes not included in prompt but is activated' D:\COMFYUI_BETA\python_embeded\python.exe D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\scripts\inference.py --config "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\tmp.yaml" --source_image "D:\COMFYUI_BETA\ComfyUI\input\unnamed.png" --driving_audio "D:\COMFYUI_BETA\ComfyUI\input\CLAIRE_YOURE_RIGHT_SOUFFLE_v01.mp3" --output D:\COMFYUI_BETA\ComfyUI\output\hallo_1718860470560910200.mp4 --pose_weight 1.0 --face_weight 1.0 --lip_weight 1.0 --face_expand_ratio 1.2 D:\COMFYUI_BETA\python_embeded\Lib\site-packages\diffusers\models\transformers\transformer_2d.py:34: FutureWarning: Transformer2DModelOutput is deprecated and will be removed in version 1.0.0. Importing Transformer2DModelOutput from diffusers.models.transformer_2d is deprecated and this will be removed in a future version. Please use from diffusers.models.modeling_outputs import Transformer2DModelOutput, instead. deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message) 2024-06-20 00:14:34.0948621 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

EP Error EP Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported. when using ['CUDAExecutionProvider', 'CPUExecutionProvider'] Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

2024-06-20 00:14:34.1682719 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Traceback (most recent call last): File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in init self._create_inference_session(providers, provider_options, disabled_optimizers) File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session sess.initialize_session(providers, provider_options, disabled_optimizers) RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\scripts\inference.py", line 372, in inference_process(command_line_args) File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\scripts\inference.py", line 156, in inference_process with ImageProcessor(img_size, face_analysis_model_path) as image_processor: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\hallo\datasets\image_processor.py", line 97, in init self.face_analysis = FaceAnalysis( ^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\app\face_analysis.py", line 31, in init model = model_zoo.get_model(onnx_file, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\model_zoo\model_zoo.py", line 96, in get_model model = router.get_model(providers=providers, provider_options=provider_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\model_zoo\model_zoo.py", line 40, in get_model session = PickableInferenceSession(self.onnx_file, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\model_zoo\model_zoo.py", line 25, in init super().init(model_path, **kwargs) File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in init raise fallback_error from e File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in init self._create_inference_session(self._fallback_providers, None) File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session sess.initialize_session(providers, provider_options, disabled_optimizers) RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

Prompt executed in 4.07 seconds

SamKhoze commented 1 week ago

Hello @rethink-studios,

I think you are getting error in different custom node "IC-Light-ComfyUI-Node", can you ask them regrading this