search

kijai / ComfyUI-MochiWrapper

Apache License 2.0
648 stars 53 forks source link

help! # ComfyUI Error Report ## Error Details Exception Type: RuntimeError ## User specified an unsupported autocast device_type 'mps' #49

Open Xyok2022 opened 3 weeks ago

Xyok2022 commented 3 weeks ago

macos [system]

ComfyUI Error Report

Error Details

## System Information
- **ComfyUI Version:** v0.2.4
- **Arguments:** main.py --use-split-cross-attention
- **OS:** posix
- **Python Version:** 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 2023, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)]
- **Embedded Python:** false
- **PyTorch Version:** 2.4.0
## Devices

- **Name:** mps
  - **Type:** mps
  - **VRAM Total:** 68719476736
  - **VRAM Free:** 35375071232
  - **Torch VRAM Total:** 68719476736
  - **Torch VRAM Free:** 35375071232

## Logs

2024-10-31 00:09:45,105 - root - INFO - Total VRAM 65536 MB, total RAM 65536 MB 2024-10-31 00:09:45,105 - root - INFO - pytorch version: 2.4.0 2024-10-31 00:09:45,105 - root - INFO - Set vram state to: SHARED 2024-10-31 00:09:45,105 - root - INFO - Device: mps 2024-10-31 00:09:45,446 - root - INFO - Using split optimization for cross attention 2024-10-31 00:09:45,858 - root - INFO - [Prompt Server] web root: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/web 2024-10-31 00:09:46,203 - numexpr.utils - INFO - NumExpr defaulting to 12 threads. 2024-10-31 00:09:47,832 - root - INFO - Total VRAM 65536 MB, total RAM 65536 MB 2024-10-31 00:09:47,832 - root - INFO - pytorch version: 2.4.0 2024-10-31 00:09:47,832 - root - INFO - Set vram state to: SHARED 2024-10-31 00:09:47,832 - root - INFO - Device: mps 2024-10-31 00:09:47,887 - root - WARNING - Traceback (most recent call last): File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/nodes.py", line 2001, in load_custom_node module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper/init.py", line 1, in from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper/nodes.py", line 12, in from .mochi_preview.t2v_synth_mochi import T2VSynthMochiModel File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper/mochi_preview/t2v_synth_mochi.py", line 23, in from .dit.joint_model.asymm_models_joint import AsymmDiTJoint File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper/mochi_preview/dit/joint_model/asymm_models_joint.py", line 49, in if torch.cuda.get_device_properties(0).major <= 7.5: File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/cuda/init.py", line 465, in get_device_properties _lazy_init() # will define _get_device_properties File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/cuda/init.py", line 305, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

2024-10-31 00:09:47,887 - root - WARNING - Cannot import /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper module for custom nodes: Torch not compiled with CUDA enabled 2024-10-31 00:09:50,521 - root - INFO - Import times for custom nodes: 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/websocket_image_save.py 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/AIGODLIKE-ComfyUI-Translation 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ControlNet-LLLite-ComfyUI 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/FreeU_Advanced 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_TiledKSampler 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_experiments 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/stability-ComfyUI-nodes 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-WD14-Tagger 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/PowerNoiseSuite 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_IPAdapter_plus 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_FizzNodes 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/images-grid-comfy-plugin 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Custom-Scripts 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_UltimateSDUpscale 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/Derfuu_ComfyUI_ModdedNodes 2024-10-31 00:09:50,521 - root - INFO - 0.0 seconds (IMPORT FAILED): /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/rgthree-comfy 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Advanced-ControlNet 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-KJNodes 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/efficiency-nodes-comfyui 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/comfyui-workspace-manager 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_Comfyroll_CustomNodes 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/comfyui_controlnet_aux 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-AnimateDiff-Evolved 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-VideoHelperSuite 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Inspire-Pack 2024-10-31 00:09:50,522 - root - INFO - 0.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper-main 2024-10-31 00:09:50,522 - root - INFO - 0.1 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Manager 2024-10-31 00:09:50,522 - root - INFO - 0.2 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Crystools 2024-10-31 00:09:50,522 - root - INFO - 0.6 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_Custom_Nodes_AlekPet 2024-10-31 00:09:50,522 - root - INFO - 1.5 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Impact-Pack 2024-10-31 00:09:50,522 - root - INFO - 2.0 seconds: /Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Marigold 2024-10-31 00:09:50,522 - root - INFO - 2024-10-31 00:09:50,529 - root - INFO - Starting server

2024-10-31 00:09:50,529 - root - INFO - To see the GUI go to: http://127.0.0.1:8188 2024-10-31 00:10:34,719 - root - INFO - got prompt 2024-10-31 00:10:34,904 - root - INFO - Requested to load SD3ClipModel_ 2024-10-31 00:10:34,904 - root - INFO - Loading 1 new model 2024-10-31 00:10:34,907 - root - INFO - loaded completely 0.0 9083.38671875 True 2024-10-31 00:11:01,656 - root - INFO - Initializing model... 2024-10-31 00:11:01,874 - root - INFO - Loading model state_dict from /Users/yx/Documents/ComfyUI-Mochi1-StarAI/models/diffusion_models/mochi/mochi_preview_dit_fp8_e4m3fn.safetensors... 2024-10-31 00:11:01,968 - root - INFO - Using accelerate to load and assign model weights to device... 2024-10-31 00:11:08,705 - root - INFO - Using cfg schedule: 0.0 2024-10-31 00:11:08,714 - root - ERROR - !!! Exception during processing !!! object of type 'float' has no len() 2024-10-31 00:11:08,714 - root - ERROR - Traceback (most recent call last): File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper-main/nodes.py", line 418, in process latents = model.run(args) File "/Users/yx/Documents/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper-main/mochi_preview/t2v_synth_mochi.py", line 241, in run len(cfg_schedule) == sample_steps TypeError: object of type 'float' has no len()

2024-10-31 00:11:08,714 - root - INFO - Prompt executed in 33.97 seconds

## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":15,"last_link_id":17,"nodes":[{"id":1,"type":"MochiTextEncode","pos":{"0":484,"1":258},"size":{"0":413.45361328125,"1":268.5947265625},"flags":{},"order":4,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":1,"label":"clip"}],"outputs":[{"name":"conditioning","type":"CONDITIONING","links":[7],"slot_index":0,"label":"conditioning"},{"name":"clip","type":"CLIP","links":null,"label":"clip"}],"properties":{"Node name for S&R":"MochiTextEncode"},"widgets_values":["An anime girl is sitting on a stool by the window reading a book",1,true,true]},{"id":8,"type":"MochiTextEncode","pos":{"0":481,"1":577},"size":{"0":400,"1":200},"flags":{},"order":5,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":8,"label":"clip"}],"outputs":[{"name":"conditioning","type":"CONDITIONING","links":[9],"slot_index":0,"label":"conditioning"},{"name":"clip","type":"CLIP","links":null,"label":"clip"}],"properties":{"Node name for S&R":"MochiTextEncode"},"widgets_values":["",1,true,true]},{"id":9,"type":"VHS_VideoCombine","pos":{"0":1683,"1":63},"size":[600.6322021484375,310],"flags":{},"order":9,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":15,"shape":7,"label":"图像"},{"name":"audio","type":"AUDIO","link":null,"shape":7,"label":"音频"},{"name":"meta_batch","type":"VHS_BatchManager","link":null,"shape":7,"label":"批次管理"},{"name":"vae","type":"VAE","link":null,"shape":7}],"outputs":[{"name":"Filenames","type":"VHS_FILENAMES","links":null,"label":"文件名"}],"properties":{"Node name for S&R":"VHS_VideoCombine"},"widgets_values":{"frame_rate":24,"loop_count":0,"filename_prefix":"视频生成\Mochi_preview","format":"video/h264-mp4","pix_fmt":"yuv420p","crf":19,"save_metadata":true,"pingpong":false,"save_output":true,"videopreview":{"hidden":false,"paused":false,"params":{"filename":"Mochi_preview_00054.mp4","subfolder":"视频生成","type":"output","format":"video/h264-mp4","frame_rate":24},"muted":false}}},{"id":11,"type":"GetImageSizeAndCount","pos":{"0":1385,"1":441},"size":{"0":222.00714111328125,"1":86},"flags":{},"order":8,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":14,"label":"图像"}],"outputs":[{"name":"image","type":"IMAGE","links":[15],"slot_index":0,"label":"图像"},{"name":"width","type":"INT","links":null,"label":"宽度"},{"name":"height","type":"INT","links":null,"label":"高度"},{"name":"count","type":"INT","links":null,"label":"数量"}],"properties":{"Node name for S&R":"GetImageSizeAndCount"},"widgets_values":[]},{"id":15,"type":"Note","pos":{"0":42,"1":-11},"size":{"0":391.9800109863281,"1":258.14373779296875},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[],"properties":{},"widgets_values":["t5xxl_fp8_e4m3fn.safetensors\n\n目录：\ComfyUI\models\clip\t5\n\n\nmochi_preview_vae_bf16.safetensors\n\n目录：\ComfyUI\models\vae\mochi\n\n\nmochi_preview_dit_bf16.safetensors|\nmochi_preview_dit_fp8_e4m3fn.safetensors|\nmochi_preview_dit_GGUF_Q4_0_v1.safetensors\n\n三选一就行\n\n目录：\ComfyUI\models\diffusion_models\mochi"],"color":"#432","bgcolor":"#653"},{"id":2,"type":"CLIPLoader","pos":{"0":82,"1":431},"size":{"0":316.0216979980469,"1":86.29476165771484},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"CLIP","type":"CLIP","links":[1,8],"label":"CLIP"}],"properties":{"Node name for S&R":"CLIPLoader"},"widgets_values":["t5xxl_fp16_2.safetensors","sd3"]},{"id":13,"type":"MochiModelLoader","pos":{"0":555,"1":-51},"size":{"0":432.56671142578125,"1":150},"flags":{},"order":2,"mode":0,"inputs":[{"name":"trigger","type":"CONDITIONING","link":null,"shape":7,"label":"trigger"},{"name":"compile_args","type":"MOCHICOMPILEARGS","link":null,"shape":7,"label":"compile_args"}],"outputs":[{"name":"mochi_model","type":"MOCHIMODEL","links":[16],"slot_index":0,"label":"mochi_model"}],"properties":{"Node name for S&R":"MochiModelLoader"},"widgets_values":["mochi/mochi_preview_dit_fp8_e4m3fn.safetensors","fp8_e4m3fn","sdpa",false]},{"id":10,"type":"MochiDecode","pos":{"0":1306,"1":158},"size":{"0":315,"1":222},"flags":{},"order":7,"mode":0,"inputs":[{"name":"vae","type":"MOCHIVAE","link":17,"label":"vae"},{"name":"samples","type":"LATENT","link":12,"label":"samples"}],"outputs":[{"name":"images","type":"IMAGE","links":[14],"slot_index":0,"label":"images"}],"properties":{"Node name for S&R":"MochiDecode"},"widgets_values":[true,true,6,160,312,0.25,0.25]},{"id":14,"type":"MochiVAELoader","pos":{"0":644,"1":134},"size":{"0":355.20001220703125,"1":58},"flags":{},"order":3,"mode":0,"inputs":[{"name":"torch_compile_args","type":"MOCHICOMPILEARGS","link":null,"shape":7,"label":"torch_compile_args"}],"outputs":[{"name":"mochi_vae","type":"MOCHIVAE","links":[17],"slot_index":0,"label":"mochi_vae"}],"properties":{"Node name for S&R":"MochiVAELoader"},"widgets_values":["sdxl_vae_fp16fix.safetensors"]},{"id":5,"type":"MochiSampler","pos":{"0":960,"1":243},"size":{"0":315,"1":286},"flags":{},"order":6,"mode":0,"inputs":[{"name":"model","type":"MOCHIMODEL","link":16,"label":"model"},{"name":"positive","type":"CONDITIONING","link":7,"label":"positive"},{"name":"negative","type":"CONDITIONING","link":9,"label":"negative"},{"name":"opt_sigmas","type":"SIGMAS","link":null,"shape":7,"label":"opt_sigmas"}],"outputs":[{"name":"samples","type":"LATENT","links":[12],"slot_index":0,"label":"samples"}],"properties":{"Node name for S&R":"MochiSampler"},"widgets_values":[360,360,13,10,1,0,"fixed",0]}],"links":[[1,2,0,1,0,"CLIP"],[7,1,0,5,1,"CONDITIONING"],[8,2,0,8,0,"CLIP"],[9,8,0,5,2,"CONDITIONING"],[12,5,0,10,1,"LATENT"],[14,10,0,11,0,"IMAGE"],[15,11,0,9,0,"IMAGE"],[16,13,0,5,0,"MOCHIMODEL"],[17,14,0,10,0,"MOCHIVAE"]],"groups":[],"config":{},"extra":{"ds":{"scale":1,"offset":[10.098185883740598,14.909050986111254]},"workspace_info":{"id":"N6IpkajuzM4xF6qgxJBi3"}},"version":0.4}



## Additional Context
<img width="1284" alt="20241031" src="https://github.com/user-attachments/assets/34b03606-ef0e-4e99-9f41-ef0cd55405e6">
kijai commented 3 weeks ago

Remake the sampler node

Xyok2022 commented 3 weeks ago

thanks [kijai]

another Error Report

ComfyUI Error Report

Error Details

## System Information
- **ComfyUI Version:** v0.2.4
- **Arguments:** main.py --use-split-cross-attention
- **OS:** posix
- **Python Version:** 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 2023, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)]
- **Embedded Python:** false
- **PyTorch Version:** 2.4.0
## Devices

- **Name:** mps
  - **Type:** mps
  - **VRAM Total:** 68719476736
  - **VRAM Free:** 12041043968
  - **Torch VRAM Total:** 68719476736
  - **Torch VRAM Free:** 12041043968

## Logs

2024-10-31 06:27:26,159 - root - INFO - Total VRAM 65536 MB, total RAM 65536 MB 2024-10-31 06:27:26,159 - root - INFO - pytorch version: 2.4.0 2024-10-31 06:27:26,159 - root - INFO - Set vram state to: SHARED 2024-10-31 06:27:26,159 - root - INFO - Device: mps 2024-10-31 06:27:26,661 - root - INFO - Using split optimization for cross attention 2024-10-31 06:27:27,706 - root - INFO - [Prompt Server] web root: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/web 2024-10-31 06:27:28,387 - numexpr.utils - INFO - NumExpr defaulting to 12 threads. 2024-10-31 06:27:29,698 - root - INFO - Total VRAM 65536 MB, total RAM 65536 MB 2024-10-31 06:27:29,698 - root - INFO - pytorch version: 2.4.0 2024-10-31 06:27:29,698 - root - INFO - Set vram state to: SHARED 2024-10-31 06:27:29,698 - root - INFO - Device: mps 2024-10-31 06:27:33,116 - root - INFO - Import times for custom nodes: 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/websocket_image_save.py 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/AIGODLIKE-ComfyUI-Translation 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/FreeU_Advanced 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ControlNet-LLLite-ComfyUI 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/stability-ComfyUI-nodes 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_experiments 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_TiledKSampler 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-WD14-Tagger 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_IPAdapter_plus 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/PowerNoiseSuite 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/images-grid-comfy-plugin 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_FizzNodes 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Custom-Scripts 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_UltimateSDUpscale 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/Derfuu_ComfyUI_ModdedNodes 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/1 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-KJNodes 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Advanced-ControlNet 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/rgthree-comfy 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/efficiency-nodes-comfyui 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/comfyui-workspace-manager 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_Comfyroll_CustomNodes 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/comfyui_controlnet_aux 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-AnimateDiff-Evolved 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-VideoHelperSuite 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Inspire-Pack 2024-10-31 06:27:33,117 - root - INFO - 0.0 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-MochiWrapper-main 2024-10-31 06:27:33,117 - root - INFO - 0.1 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Manager 2024-10-31 06:27:33,117 - root - INFO - 0.2 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Crystools 2024-10-31 06:27:33,117 - root - INFO - 0.8 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI_Custom_Nodes_AlekPet 2024-10-31 06:27:33,117 - root - INFO - 1.2 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Impact-Pack 2024-10-31 06:27:33,117 - root - INFO - 2.5 seconds: /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/ComfyUI-Marigold 2024-10-31 06:27:33,117 - root - INFO - 2024-10-31 06:27:33,124 - root - INFO - Starting server

2024-10-31 06:27:33,125 - root - INFO - To see the GUI go to: http://127.0.0.1:8188 2024-10-31 06:28:19,284 - root - INFO - got prompt 2024-10-31 06:28:19,480 - root - INFO - Requested to load SD3ClipModel_ 2024-10-31 06:28:19,481 - root - INFO - Loading 1 new model 2024-10-31 06:28:19,484 - root - INFO - loaded completely 0.0 9083.38671875 True 2024-10-31 06:28:44,930 - root - INFO - Initializing model... 2024-10-31 06:28:45,020 - root - INFO - Loading model state_dict from /Users/yx/Downloads/ComfyUI-Mochi1-StarAI/models/diffusion_models/mochi/mochi_preview_dit_fp8_e4m3fn.safetensors... 2024-10-31 06:28:45,122 - root - INFO - Using accelerate to load and assign model weights to device... 2024-10-31 06:29:02,774 - root - ERROR - !!! Exception during processing !!! User specified an unsupported autocast device_type 'mps' 2024-10-31 06:29:02,775 - root - ERROR - Traceback (most recent call last): File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/1/nodes.py", line 418, in process latents = model.run(args) File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/1/mochi_preview/t2v_synth_mochi.py", line 344, in run pred = model_fn( File "/Users/yx/Downloads/ComfyUI-Mochi1-StarAI/custom_nodes/1/mochi_preview/t2v_synth_mochi.py", line 328, in model_fn with torch.autocast(mm.get_autocast_device(self.device), dtype=autocast_dtype): File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 238, in init raise RuntimeError( RuntimeError: User specified an unsupported autocast device_type 'mps'

2024-10-31 06:29:02,778 - root - INFO - Prompt executed in 43.47 seconds

## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":17,"last_link_id":25,"nodes":[{"id":1,"type":"MochiTextEncode","pos":{"0":484,"1":258},"size":{"0":413.45361328125,"1":268.5947265625},"flags":{},"order":4,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":1,"label":"clip"}],"outputs":[{"name":"conditioning","type":"CONDITIONING","links":[23],"slot_index":0,"label":"conditioning"},{"name":"clip","type":"CLIP","links":null,"label":"clip"}],"properties":{"Node name for S&R":"MochiTextEncode"},"widgets_values":["An anime girl is sitting on a stool by the window reading a book",1,true,true]},{"id":8,"type":"MochiTextEncode","pos":{"0":481,"1":577},"size":{"0":400,"1":200},"flags":{},"order":5,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":8,"label":"clip"}],"outputs":[{"name":"conditioning","type":"CONDITIONING","links":[24],"slot_index":0,"label":"conditioning"},{"name":"clip","type":"CLIP","links":null,"label":"clip"}],"properties":{"Node name for S&R":"MochiTextEncode"},"widgets_values":["",1,true,true]},{"id":9,"type":"VHS_VideoCombine","pos":{"0":1683,"1":63},"size":[600.6322021484375,310],"flags":{},"order":9,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":15,"shape":7,"label":"图像"},{"name":"audio","type":"AUDIO","link":null,"shape":7,"label":"音频"},{"name":"meta_batch","type":"VHS_BatchManager","link":null,"shape":7,"label":"批次管理"},{"name":"vae","type":"VAE","link":null,"shape":7}],"outputs":[{"name":"Filenames","type":"VHS_FILENAMES","links":null,"label":"文件名"}],"properties":{"Node name for S&R":"VHS_VideoCombine"},"widgets_values":{"frame_rate":24,"loop_count":0,"filename_prefix":"视频生成\Mochi_preview","format":"video/h264-mp4","pix_fmt":"yuv420p","crf":19,"save_metadata":true,"pingpong":false,"save_output":true,"videopreview":{"hidden":false,"paused":false,"params":{"filename":"Mochi_preview_00054.mp4","subfolder":"视频生成","type":"output","format":"video/h264-mp4","frame_rate":24},"muted":false}}},{"id":11,"type":"GetImageSizeAndCount","pos":{"0":1385,"1":441},"size":{"0":222.00714111328125,"1":86},"flags":{},"order":8,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":14,"label":"图像"}],"outputs":[{"name":"image","type":"IMAGE","links":[15],"slot_index":0,"label":"图像"},{"name":"width","type":"INT","links":null,"label":"宽度"},{"name":"height","type":"INT","links":null,"label":"高度"},{"name":"count","type":"INT","links":null,"label":"数量"}],"properties":{"Node name for S&R":"GetImageSizeAndCount"},"widgets_values":[]},{"id":15,"type":"Note","pos":{"0":42,"1":-11},"size":{"0":391.9800109863281,"1":258.14373779296875},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[],"properties":{},"widgets_values":["t5xxl_fp8_e4m3fn.safetensors\n\n目录：\ComfyUI\models\clip\t5\n\n\nmochi_preview_vae_bf16.safetensors\n\n目录：\ComfyUI\models\vae\mochi\n\n\nmochi_preview_dit_bf16.safetensors|\nmochi_preview_dit_fp8_e4m3fn.safetensors|\nmochi_preview_dit_GGUF_Q4_0_v1.safetensors\n\n三选一就行\n\n目录：\ComfyUI\models\diffusion_models\mochi"],"color":"#432","bgcolor":"#653"},{"id":2,"type":"CLIPLoader","pos":{"0":82,"1":431},"size":{"0":316.0216979980469,"1":86.29476165771484},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"CLIP","type":"CLIP","links":[1,8],"label":"CLIP"}],"properties":{"Node name for S&R":"CLIPLoader"},"widgets_values":["t5xxl_fp16_2.safetensors","sd3"]},{"id":13,"type":"MochiModelLoader","pos":{"0":555,"1":-51},"size":{"0":432.56671142578125,"1":150},"flags":{},"order":2,"mode":0,"inputs":[{"name":"trigger","type":"CONDITIONING","link":null,"shape":7,"label":"trigger"},{"name":"compile_args","type":"MOCHICOMPILEARGS","link":null,"shape":7,"label":"compile_args"}],"outputs":[{"name":"mochi_model","type":"MOCHIMODEL","links":[22],"slot_index":0,"label":"mochi_model"}],"properties":{"Node name for S&R":"MochiModelLoader"},"widgets_values":["mochi/mochi_preview_dit_fp8_e4m3fn.safetensors","fp8_e4m3fn","sdpa",false]},{"id":10,"type":"MochiDecode","pos":{"0":1306,"1":158},"size":{"0":315,"1":222},"flags":{},"order":7,"mode":0,"inputs":[{"name":"vae","type":"MOCHIVAE","link":17,"label":"vae"},{"name":"samples","type":"LATENT","link":25,"label":"samples"}],"outputs":[{"name":"images","type":"IMAGE","links":[14],"slot_index":0,"label":"images"}],"properties":{"Node name for S&R":"MochiDecode"},"widgets_values":[true,true,6,160,312,0.25,0.25]},{"id":14,"type":"MochiVAELoader","pos":{"0":644,"1":134},"size":{"0":355.20001220703125,"1":58},"flags":{},"order":3,"mode":0,"inputs":[{"name":"torch_compile_args","type":"MOCHICOMPILEARGS","link":null,"shape":7,"label":"torch_compile_args"}],"outputs":[{"name":"mochi_vae","type":"MOCHIVAE","links":[17],"slot_index":0,"label":"mochi_vae"}],"properties":{"Node name for S&R":"MochiVAELoader"},"widgets_values":["sdxl_vae_fp16fix.safetensors"]},{"id":17,"type":"MochiSampler","pos":{"0":960,"1":243},"size":[315,286],"flags":{},"order":6,"mode":0,"inputs":[{"name":"model","type":"MOCHIMODEL","link":22,"label":"model"},{"name":"positive","type":"CONDITIONING","link":23,"label":"positive"},{"name":"negative","type":"CONDITIONING","link":24,"label":"negative"},{"name":"opt_sigmas","type":"SIGMAS","link":null,"shape":7,"label":"opt_sigmas"},{"name":"cfg_schedule","type":"FLOAT","link":null,"widget":{"name":"cfg_schedule"},"shape":7}],"outputs":[{"name":"samples","type":"LATENT","links":[25],"label":"samples"}],"properties":{"Node name for S&R":"MochiSampler"},"widgets_values":[848,480,49,50,4.5,180902137532908,"randomize",0]}],"links":[[1,2,0,1,0,"CLIP"],[8,2,0,8,0,"CLIP"],[14,10,0,11,0,"IMAGE"],[15,11,0,9,0,"IMAGE"],[17,14,0,10,0,"MOCHIVAE"],[22,13,0,17,0,"MOCHIMODEL"],[23,1,0,17,1,"CONDITIONING"],[24,8,0,17,2,"CONDITIONING"],[25,17,0,10,1,"LATENT"]],"groups":[],"config":{},"extra":{"ds":{"scale":0.5644739300537774,"offset":[193.73566912374076,390.3417317961107]},"workspace_info":{"id":"N6IpkajuzM4xF6qgxJBi3"}},"version":0.4}



## Additional Context
(Please add any additional context or steps to reproduce the error here)
YAY-3M-TA3 commented 3 weeks ago

For MPS, you'll need to use Torch nightly (2.6.0) - that version can autocast to MPS and bfloat16 is supported.
I've also made changes to support MPS:

Nodes.py: line 30: + os.environ["TOKENIZERS_PARALLELISM"] = "false"

line 185: @niw

- #set_module_tensor_to_device(vae, key, dtype=torch.float32, device=device, value=vae_sd[key])
+ set_module_tensor_to_device(vae, key, dtype=torch.bfloat16, device=device, value=vae_sd[key])

line 636: + vae = vae.to(torch.bfloat16)

line 691: + vae = vae.to(torch.bfloat16)

line 693:

- with torch.autocast(mm.get_autocast_device(device), dtype=vae.dtype):
+ with torch.autocast(mm.get_autocast_device(device), dtype=torch.bfloat16)

vae/model.py line 578 + x = x.to(torch.bfloat16)

Inference and VAE will take 27Gb... with 4x4 tiling and 7 frames

(I'm trying to get this under 23Gb... I've been able to do this with PyramidFlow's tiled vae with @niw, but I'd like to see if that can be done for mochi...)

kijai commented 3 weeks ago

There is also the standalone model loading nodes, the VAE loader allows setting the precision:

image

The download loader shouldn't have loaded it in fp32, that was oversight from me and I changed it now.

cchance27 commented 3 weeks ago

when i use the Down(load) it fails even though its set to fp16, the standalone does indeed work properly when set to fp16., or atleast it seems to be its running right now will know shortly, well if i can stop it from OOM'ing lol 32gb seems to be ... an issue lol

niw commented 3 weeks ago

dtype mismatch is in fact a problem yet it's more complicated on mps to run this node. At this moment, the latest main (with #66) and #67 patch, with enough memory such as 64GB, it can genrate video on MacBook Pro properly.

A few confusing problem was existing in pytorch:

With all these problems, it was probably very difficult to run this custom node on Mac.

YAY-3M-TA3 commented 2 weeks ago

With all these problems, it was probably very difficult to run this custom node on Mac.

I can run on Mac, but it hits my limit (it needs 27GB, i only have 24Gb). what I did

emilune commented 2 weeks ago

With all these problems, it was probably very difficult to run this custom node on Mac.

I can run on Mac, but it hits my limit (it needs 27GB, i only have 24Gb). what I did

Hello, I'm a newbie with terminal, can you upload your modified Nodes.py and Vae/Model.py files ? It will be easier to test. Thanks in advance