kijai / ComfyUI-PyramidFlowWrapper

MIT License
196 stars 8 forks source link

PyramidFlowSampler Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead. #29

Open john2stai opened 3 days ago

john2stai commented 3 days ago

I kept getting this error when I chose fp32 for dtypes on the model loader node.

ComfyUI Error Report

Error Details

## System Information
- **ComfyUI Version:** v0.2.3-5-g3c60ecd
- **Arguments:** main.py --use-split-cross-attention --force-fp16
- **OS:** posix
- **Python Version:** 3.12.5 (main, Aug  6 2024, 19:08:49) [Clang 15.0.0 (clang-1500.3.9.4)]
- **Embedded Python:** false
- **PyTorch Version:** 2.4.1
## Devices

- **Name:** mps
  - **Type:** mps
  - **VRAM Total:** 38654705664
  - **VRAM Free:** 9696903168
  - **Torch VRAM Total:** 38654705664
  - **Torch VRAM Free:** 9696903168

## Logs

2024-10-16 15:51:44,222 - root - INFO - Total VRAM 36864 MB, total RAM 36864 MB 2024-10-16 15:51:44,222 - root - INFO - pytorch version: 2.4.1 2024-10-16 15:51:44,222 - root - INFO - Forcing FP16. 2024-10-16 15:51:44,222 - root - INFO - Set vram state to: SHARED 2024-10-16 15:51:44,222 - root - INFO - Device: mps 2024-10-16 15:51:44,651 - root - INFO - Using split optimization for cross attention 2024-10-16 15:51:45,313 - root - INFO - [Prompt Server] web root: /Users/john2stai/Desktop/sd/ComfyUI/web 2024-10-16 15:51:54,483 - root - INFO - Total VRAM 36864 MB, total RAM 36864 MB 2024-10-16 15:51:54,483 - root - INFO - pytorch version: 2.4.1 2024-10-16 15:51:54,483 - root - INFO - Forcing FP16. 2024-10-16 15:51:54,483 - root - INFO - Set vram state to: SHARED 2024-10-16 15:51:54,483 - root - INFO - Device: mps 2024-10-16 15:51:55,770 - root - WARNING - Traceback (most recent call last): File "/Users/john2stai/Desktop/sd/ComfyUI/nodes.py", line 2001, in load_custom_node module_spec.loader.exec_module(module) File "", line 995, in exec_module File "", line 488, in _call_with_frames_removed File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/init.py", line 2, in from .src.nodes_mappings import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/src/nodes_mappings.py", line 1, in from .nodes.pipeline_loader import PipelineLoader File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/src/nodes/pipeline_loader.py", line 9, in from ..idm_vton.unet_hacked_tryon import UNet2DConditionModel File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/src/idm_vton/unet_hacked_tryon.py", line 50, in from .unet_block_hacked_tryon import ( File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/src/idm_vton/unet_block_hacked_tryon.py", line 25, in from diffusers.models.dual_transformer_2d import DualTransformer2DModel ModuleNotFoundError: No module named 'diffusers.models.dual_transformer_2d'

2024-10-16 15:51:55,770 - root - WARNING - Cannot import /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON module for custom nodes: No module named 'diffusers.models.dual_transformer_2d' 2024-10-16 15:51:55,771 - root - INFO - Import times for custom nodes: 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/websocket_image_save.py 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/AIGODLIKE-COMFYUI-TRANSLATION 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui-saveimage-plus 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Miaoshouai-Tagger 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/cg-use-everywhere 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Logic 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Molmo 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Inpaint-CropAndStitch 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_Noise 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/masquerade-nodes-comfyui 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/Comfyui_TTP_Toolset 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-post-processing-nodes 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Lora-Auto-Trigger-Words 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-enricos-nodes 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-WD14-Tagger 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/pre_cfg_comfy_nodes_for_ComfyUI 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui_face_parsing 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-TiledDiffusion 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui-lama-remover 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/cg-mixed-casting 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-moondream 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ControlAltAI-Nodes 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IC-Light 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Depth-Pro 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-DDColor 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Video-Matting 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-AutoCropFaces 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_essentials 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_UltimateSDUpscale 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/Comfy_KepListStuff 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui-portrait-master 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/Derfuu_ComfyUI_ModdedNodes 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-AdvancedLivePortrait 2024-10-16 15:51:55,771 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/OneButtonPrompt 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui-browser 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/efficiency-nodes-comfyui 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-KJNodes 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds (IMPORT FAILED): /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-IDM-VTON 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/rgthree-comfy 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-GGUF 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_tinyterraNodes 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui_controlnet_aux 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_Searge_LLM 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_NYJY 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/Comfyui-ergouzi-Nodes 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Inspire-Pack 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-CCSR 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Florence2 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-MingNodes 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfy_mtb 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-segment-anything-2 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Impact-Pack 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PuLID-Flux 2024-10-16 15:51:55,772 - root - INFO - 0.0 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Long-CLIP 2024-10-16 15:51:55,772 - root - INFO - 0.1 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-SUPIR 2024-10-16 15:51:55,772 - root - INFO - 0.1 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Manager 2024-10-16 15:51:55,772 - root - INFO - 0.1 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_LayerStyle 2024-10-16 15:51:55,772 - root - INFO - 0.1 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Easy-Use 2024-10-16 15:51:55,772 - root - INFO - 0.1 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-BrushNet 2024-10-16 15:51:55,772 - root - INFO - 0.2 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-Allor 2024-10-16 15:51:55,772 - root - INFO - 0.2 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/eden_comfy_pipelines 2024-10-16 15:51:55,772 - root - INFO - 0.3 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_Stable_Makeup 2024-10-16 15:51:55,772 - root - INFO - 0.5 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI_FaceAnalysis 2024-10-16 15:51:55,772 - root - INFO - 0.6 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/comfyui-reactor-node 2024-10-16 15:51:55,772 - root - INFO - 0.7 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/was-node-suite-comfyui 2024-10-16 15:51:55,772 - root - INFO - 3.4 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PMRF 2024-10-16 15:51:55,772 - root - INFO - 3.4 seconds: /Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/DZ-FaceDetailer 2024-10-16 15:51:55,772 - root - INFO - 2024-10-16 15:51:55,778 - root - INFO - Starting server

2024-10-16 15:51:55,778 - root - INFO - To see the GUI go to: http://127.0.0.1:8188 2024-10-16 15:52:01,911 - root - INFO - got prompt 2024-10-16 15:52:25,957 - root - ERROR - !!! Exception during processing !!! User specified an unsupported autocast device_type 'mps' 2024-10-16 15:52:25,959 - root - ERROR - Traceback (most recent call last): File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 260, in sample autocast_context = torch.autocast(mm.get_autocast_device(device), dtype=model["text_encoder_dtype"]) if autocastcondition else nullcontext() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/amp/autocast_mode.py", line 238, in init raise RuntimeError( RuntimeError: User specified an unsupported autocast device_type 'mps'

2024-10-16 15:52:25,960 - root - INFO - Prompt executed in 24.04 seconds 2024-10-16 15:55:06,095 - root - INFO - got prompt 2024-10-16 15:58:05,179 - root - ERROR - !!! Exception during processing !!! Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead. 2024-10-16 15:58:05,186 - root - ERROR - Traceback (most recent call last): File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 213, in sample latents = model["model"].generate_i2v( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 434, in generate_i2v intermed_latents = self.generate_one_unit( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 227, in generate_one_unit self.scheduler.set_timesteps(num_inference_steps[i_s], i_s, device=device) File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/diffusion_schedulers/scheduling_flow_matching.py", line 195, in set_timesteps self.timesteps = torch.from_numpy(timesteps).to(device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

2024-10-16 15:58:05,191 - root - INFO - Prompt executed in 179.08 seconds 2024-10-16 16:00:00,767 - root - INFO - got prompt 2024-10-16 16:00:01,146 - root - ERROR - !!! Exception during processing !!! Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead. 2024-10-16 16:00:01,147 - root - ERROR - Traceback (most recent call last): File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 213, in sample latents = model["model"].generate_i2v( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 434, in generate_i2v intermed_latents = self.generate_one_unit( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 227, in generate_one_unit self.scheduler.set_timesteps(num_inference_steps[i_s], i_s, device=device) File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/diffusion_schedulers/scheduling_flow_matching.py", line 195, in set_timesteps self.timesteps = torch.from_numpy(timesteps).to(device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

2024-10-16 16:00:01,147 - root - INFO - Prompt executed in 0.37 seconds 2024-10-16 16:00:09,105 - root - INFO - got prompt 2024-10-16 16:00:09,346 - root - ERROR - !!! Exception during processing !!! Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead. 2024-10-16 16:00:09,347 - root - ERROR - Traceback (most recent call last): File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 213, in sample latents = model["model"].generate_i2v( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 434, in generate_i2v intermed_latents = self.generate_one_unit( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 227, in generate_one_unit self.scheduler.set_timesteps(num_inference_steps[i_s], i_s, device=device) File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/diffusion_schedulers/scheduling_flow_matching.py", line 195, in set_timesteps self.timesteps = torch.from_numpy(timesteps).to(device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

2024-10-16 16:00:09,347 - root - INFO - Prompt executed in 0.23 seconds 2024-10-16 16:00:11,897 - root - INFO - got prompt 2024-10-16 16:00:12,125 - root - ERROR - !!! Exception during processing !!! Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead. 2024-10-16 16:00:12,126 - root - ERROR - Traceback (most recent call last): File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/john2stai/Desktop/sd/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 213, in sample latents = model["model"].generate_i2v( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 434, in generate_i2v intermed_latents = self.generate_one_unit( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 227, in generate_one_unit self.scheduler.set_timesteps(num_inference_steps[i_s], i_s, device=device) File "/Users/john2stai/Desktop/sd/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/diffusion_schedulers/scheduling_flow_matching.py", line 195, in set_timesteps self.timesteps = torch.from_numpy(timesteps).to(device=device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

2024-10-16 16:00:12,126 - root - INFO - Prompt executed in 0.22 seconds

## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":8,"last_link_id":12,"nodes":[{"id":3,"type":"PyramidFlowVAEEncode","pos":{"0":80,"1":450},"size":{"0":280,"1":50},"flags":{},"order":3,"mode":0,"inputs":[{"name":"model","type":"PYRAMIDFLOWMODEL","link":7,"label":"model"},{"name":"image","type":"IMAGE","link":8,"label":"image"}],"outputs":[{"name":"samples","type":"LATENT","links":[6],"label":"samples","slot_index":0}],"properties":{"Node name for S&R":"PyramidFlowVAEEncode"}},{"id":7,"type":"easy imageSize","pos":{"0":-270,"1":180},"size":{"0":210,"1":100},"flags":{},"order":4,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":9,"label":"image"}],"outputs":[{"name":"width_int","type":"INT","links":[10],"label":"width_int","slot_index":0},{"name":"height_int","type":"INT","links":[11],"label":"height_int","slot_index":1}],"properties":{"Node name for S&R":"easy imageSize"},"widgets_values":["Width: 1024 , Height: 792"]},{"id":8,"type":"PreviewImage","pos":{"0":1220,"1":160},"size":{"0":210,"1":30},"flags":{},"order":7,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":12,"label":"images"}],"outputs":[],"properties":{"Node name for S&R":"PreviewImage"}},{"id":2,"type":"PyramidFlowVAEDecode","pos":{"0":830,"1":160},"size":{"0":320,"1":100},"flags":{},"order":6,"mode":0,"inputs":[{"name":"model","type":"PYRAMIDFLOWMODEL","link":2,"label":"model"},{"name":"samples","type":"LATENT","link":3,"label":"samples"}],"outputs":[{"name":"images","type":"IMAGE","links":[12],"label":"images","slot_index":0}],"properties":{"Node name for S&R":"PyramidFlowVAEDecode"},"widgets_values":[256,2]},{"id":4,"type":"PyramidFlowTextEncode","pos":{"0":20,"1":580},"size":{"0":400,"1":200},"flags":{},"order":2,"mode":0,"inputs":[{"name":"model","type":"PYRAMIDFLOWMODEL","link":4,"label":"model"},{"name":"prev_prompt","type":"PYRAMIDFLOWPROMPT","link":null,"shape":7,"label":"prev_prompt"}],"outputs":[{"name":"prompt_embeds","type":"PYRAMIDFLOWPROMPT","links":[5],"label":"prompt_embeds","slot_index":0}],"properties":{"Node name for S&R":"PyramidFlowTextEncode"},"widgets_values":["hyper quality, Ultra HD, 8K, a wizard cat, waving his magic wand, and casting spells","",true]},{"id":1,"type":"PyramidFlowSampler","pos":{"0":439.09375,"1":156.41796875},"size":[320,310],"flags":{},"order":5,"mode":0,"inputs":[{"name":"model","type":"PYRAMIDFLOWMODEL","link":1,"label":"model"},{"name":"prompt_embeds","type":"PYRAMIDFLOWPROMPT","link":5,"label":"prompt_embeds"},{"name":"input_latent","type":"LATENT","link":6,"shape":7,"label":"input_latent"},{"name":"width","type":"INT","link":10,"widget":{"name":"width"}},{"name":"height","type":"INT","link":11,"widget":{"name":"height"}}],"outputs":[{"name":"model","type":"PYRAMIDFLOWMODEL","links":[2],"label":"model","slot_index":0},{"name":"samples","type":"LATENT","links":[3],"label":"samples","slot_index":1}],"properties":{"Node name for S&R":"PyramidFlowSampler"},"widgets_values":[640,384,"10, 10, 10","10, 10, 10",8,9,5,302897501843326,"randomize",true]},{"id":5,"type":"DownloadAndLoadPyramidFlowModel","pos":{"0":50,"1":160},"size":{"0":350,"1":180},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"pyramidflow_model","type":"PYRAMIDFLOWMODEL","links":[1,4,7],"label":"pyramidflow_model","slot_index":0}],"properties":{"Node name for S&R":"DownloadAndLoadPyramidFlowModel"},"widgets_values":["rain1011/pyramid-flow-sd3","diffusion_transformer_768p","fp32","fp32","fp32",false]},{"id":6,"type":"LoadImage","pos":{"0":-370,"1":320},"size":[320,310],"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[8,9],"label":"IMAGE","slot_index":0},{"name":"MASK","type":"MASK","links":null,"label":"MASK"}],"properties":{"Node name for S&R":"LoadImage"},"widgets_values":["ComfyUI_temp_dlbla00002.png","image"]}],"links":[[1,5,0,1,0,"PYRAMIDFLOWMODEL"],[2,1,0,2,0,"PYRAMIDFLOWMODEL"],[3,1,1,2,1,"LATENT"],[4,5,0,4,0,"PYRAMIDFLOWMODEL"],[5,4,0,1,1,"PYRAMIDFLOWPROMPT"],[6,3,0,1,2,"LATENT"],[7,5,0,3,0,"PYRAMIDFLOWMODEL"],[8,6,0,3,1,"IMAGE"],[9,6,0,7,0,"IMAGE"],[10,7,0,1,3,"INT"],[11,7,1,1,4,"INT"],[12,2,0,8,0,"IMAGE"]],"groups":[],"config":{},"extra":{"ds":{"scale":1,"offset":[213.1215259577259,-220.50236640433837]}},"version":0.4}



## Additional Context
(Please add any additional context or steps to reproduce the error here)
haiggoh commented 3 days ago

Same here with pytorch version: 2.6.0.dev20241015, and I'm booting comfyui with PYTORCH_ENABLE_MPS_FALLBACK=1 PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 ./venv/bin/python main.py --use-split-cross-attention

my error message:

huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
    - Avoid using `tokenizers` before the fork if possible
    - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
got prompt
  0%|                                                    | 0/16 [00:00<?, ?it/s]
!!! Exception during processing !!! Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
Traceback (most recent call last):
  File "/Users/bra0002h/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/Users/bra0002h/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 213, in sample
    latents = model["model"].generate_i2v(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 434, in generate_i2v
    intermed_latents = self.generate_one_unit(
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bra0002h/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/pyramid_dit_for_video_gen_pipeline.py", line 227, in generate_one_unit
    self.scheduler.set_timesteps(num_inference_steps[i_s], i_s, device=device)
  File "/Users/bra0002h/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/diffusion_schedulers/scheduling_flow_matching.py", line 195, in set_timesteps
    self.timesteps = torch.from_numpy(timesteps).to(device=device)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

I tried using any of the provided model types in the checkpoint (none of which are supposed to be fp64 anyway?) but for some reason this error message keeps showing

kijai commented 3 days ago

The original code does not take mps into account, and mps does not support fp64, which is used somewhere in the calculations.

This would take work to track down every incompatible bit, and even then it would be extremely slow on mps. As I don't have a Mac anyway, this won't be something I will be doing myself.

haiggoh commented 3 days ago

Thanks for the quick heads up! I was really confused why it said fp64 when the models in the checkpoint aren't, but if it's used somewhere else I can see how that'll be tricky to track down.

My understanding of Python is unfortunately way too limited to try and contribute a fix or anything but I hope someone with will chime in and save the day for us some day. Either here or in the original project, fingers crossed!

haiggoh commented 3 days ago

things are moving incredibly fast. Somebody already committed a fix on the original project

MPS backend Thanks to @niw, Apple Silicon users (e.g. MacBook Pro with M2 24GB) can also try our model using the MPS backend! Please see https://github.com/jy0205/Pyramid-Flow/pull/108 for the details.