smthemex / ComfyUI_StoryDiffusion

You can using StoryDiffusion in ComfyUI
Apache License 2.0
241 stars 195 forks source link

MAC M1 AssertionError: Torch not compiled with CUDA enabled #47

Closed xiaoyu9308 closed 2 months ago

xiaoyu9308 commented 3 months ago

你好,请问能支持下MAC M1没有CUDA的情况吗? File "/Users/Documents/Github/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/utils/pipeline.py", line 123, in load_photomaker_adapter id_encoder = id_encoder.to("cuda", dtype=self.unet.dtype)

将 ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/utils/pipeline.py line 123 更换为 id_encoder = id_encoder.to("mps", dtype=self.unet.dtype) 或 id_encoder = id_encoder.to("cpu", dtype=self.unet.dtype) 报错

File "/Users/Documents/Github/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 1387, in story_sampler image=kwargs["image"]


KeyError: 'image'

改
Storydiffusion_Model_Loader 的 model_type 为txt2img

报错
  File "/Users/Documents/Github/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 809, in process_generation
    id_images = pipe(
                ^^^^^
  File "/opt/miniconda3/lib/python3.12/site-packages/diffusers/utils/torch_utils.py", line 80, in randn_tensor
    latents = torch.randn(shape, generator=generator, device=rand_device, dtype=dtype, layout=layout).to(device)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Placeholder storage has not been allocated on MPS device!
smthemex commented 3 months ago

手上没有M1测试呢,这里你改成,id_encoder = id_encoder.to(self.device, dtype=self.unet.dtype) 试试

xiaoyu9308 commented 3 months ago

试了下不行,报了新的错误,奈何没经验不知道怎么改

itsFrankenSense commented 2 months ago

Made the change to id_encoder and now getting: "Placeholder storage has not been allocated on MPS device!" error

Any help would be appreciated.

ComfyUI Error Report

Error Details

## System Information
- **ComfyUI Version:** v0.2.2-1-gc27ebeb
- **Arguments:** main.py --force-upcast-attention
- **OS:** posix
- **Python Version:** 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)]
- **Embedded Python:** false
- **PyTorch Version:** 2.4.1
## Devices

- **Name:** mps
  - **Type:** mps
  - **VRAM Total:** 68719476736
  - **VRAM Free:** 44823904256
  - **Torch VRAM Total:** 68719476736
  - **Torch VRAM Free:** 44823904256

## Logs

2024-09-07 20:15:16,798 - root - INFO - Total VRAM 65536 MB, total RAM 65536 MB 2024-09-07 20:15:16,798 - root - INFO - pytorch version: 2.4.1 2024-09-07 20:15:16,798 - root - INFO - Set vram state to: SHARED 2024-09-07 20:15:16,798 - root - INFO - Device: mps 2024-09-07 20:15:17,133 - root - INFO - Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --use-split-cross-attention 2024-09-07 20:15:17,443 - root - INFO - [Prompt Server] web root: /Users/franken/AI/ComfyUI/web 2024-09-07 20:15:17,866 - root - WARNING - Traceback (most recent call last): File "/Users/franken/AI/ComfyUI/nodes.py", line 1993, in load_custom_node module_spec.loader.exec_module(module) File "", line 879, in exec_module File "", line 1016, in get_code File "", line 1073, in get_data FileNotFoundError: [Errno 2] No such file or directory: '/Users/franken/AI/ComfyUI/custom_nodes/ComfyUI/init.py'

2024-09-07 20:15:17,866 - root - WARNING - Cannot import /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI module for custom nodes: [Errno 2] No such file or directory: '/Users/franken/AI/ComfyUI/custom_nodes/ComfyUI/init.py' 2024-09-07 20:15:18,211 - root - INFO - Import times for custom nodes: 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/websocket_image_save.py 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds (IMPORT FAILED): /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyMath 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_UltimateSDUpscale 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/Derfuu_ComfyUI_ModdedNodes 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/rgthree-comfy 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/efficiency-nodes-comfyui 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/x-flux-comfyui 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_essentials 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes 2024-09-07 20:15:18,211 - root - INFO - 0.0 seconds: /Users/franken/AI/ComfyUI/custom_nodes/comfyui_controlnet_aux 2024-09-07 20:15:18,211 - root - INFO - 0.1 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI-Manager 2024-09-07 20:15:18,211 - root - INFO - 0.2 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion 2024-09-07 20:15:18,211 - root - INFO - 0.3 seconds: /Users/franken/AI/ComfyUI/custom_nodes/ComfyUI-Impact-Pack 2024-09-07 20:15:18,211 - root - INFO - 2024-09-07 20:15:18,217 - root - INFO - Starting server

2024-09-07 20:15:18,217 - root - INFO - To see the GUI go to: http://127.0.0.1:8188 2024-09-07 20:15:22,829 - root - INFO - got prompt 2024-09-07 20:15:28,032 - root - ERROR - !!! Exception during processing !!! Placeholder storage has not been allocated on MPS device! 2024-09-07 20:15:28,032 - root - ERROR - Traceback (most recent call last): File "/Users/franken/AI/ComfyUI/execution.py", line 317, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/franken/AI/ComfyUI/execution.py", line 192, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/franken/AI/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/franken/AI/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(*inputs)) File "/Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 1853, in story_sampler for value in gen: File "/Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 862, in process_generation id_images = pipe( File "/Users/franken/AI/venv310/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, **kwargs) File "/Users/franken/AI/venv310/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 1100, in call latents = self.prepare_latents( File "/Users/franken/AI/venv310/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 709, in prepare_latents latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype) File "/Users/franken/AI/venv310/lib/python3.10/site-packages/diffusers/utils/torch_utils.py", line 81, in randn_tensor latents = torch.randn(shape, generator=generator, device=rand_device, dtype=dtype, layout=layout).to(device) RuntimeError: Placeholder storage has not been allocated on MPS device!

2024-09-07 20:15:28,033 - root - INFO - Prompt executed in 5.20 seconds 2024-09-07 20:46:10,525 - root - INFO - got prompt 2024-09-07 20:46:20,595 - root - ERROR - !!! Exception during processing !!! Placeholder storage has not been allocated on MPS device! 2024-09-07 20:46:20,595 - root - ERROR - Traceback (most recent call last): File "/Users/franken/AI/ComfyUI/execution.py", line 317, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/franken/AI/ComfyUI/execution.py", line 192, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/Users/franken/AI/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/Users/franken/AI/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(*inputs)) File "/Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 1853, in story_sampler for value in gen: File "/Users/franken/AI/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 862, in process_generation id_images = pipe( File "/Users/franken/AI/venv310/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, **kwargs) File "/Users/franken/AI/venv310/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 1100, in call latents = self.prepare_latents( File "/Users/franken/AI/venv310/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 709, in prepare_latents latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype) File "/Users/franken/AI/venv310/lib/python3.10/site-packages/diffusers/utils/torch_utils.py", line 81, in randn_tensor latents = torch.randn(shape, generator=generator, device=rand_device, dtype=dtype, layout=layout).to(device) RuntimeError: Placeholder storage has not been allocated on MPS device!

2024-09-07 20:46:20,595 - root - INFO - Prompt executed in 10.06 seconds

## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":19,"last_link_id":27,"nodes":[{"id":12,"type":"PreviewImage","pos":{"0":1333,"1":-174},"size":{"0":552.12255859375,"1":711.5633544921875},"flags":{},"order":2,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":26,"label":"images"}],"outputs":[],"properties":{"Node name for S&R":"PreviewImage"}},{"id":19,"type":"Storydiffusion_Sampler","pos":{"0":839,"1":-287},"size":{"0":460.9025573730469,"1":762.0733032226562},"flags":{},"order":1,"mode":0,"inputs":[{"name":"model","type":"STORY_DICT","link":24,"label":"model"},{"name":"image","type":"IMAGE","link":null,"label":"image"},{"name":"control_image","type":"IMAGE","link":null,"label":"control_image"}],"outputs":[{"name":"image","type":"IMAGE","links":[26],"slot_index":0,"shape":3,"label":"image"},{"name":"prompt_array","type":"STRING","links":null,"shape":3,"label":"prompt_array"}],"properties":{"Node name for S&R":"Storydiffusion_Sampler"},"widgets_values":["[Taylor] a woman img, wearing a white T-shirt, blue loose hair.\n[Lecun] a man img,wearing a suit,black hair.","[Taylor]wake up in the bed,medium shot;\n[Taylor]have breakfast by the window;\n[Lecun] drving on the road,medium shot;\n[Taylor] is sleeping,[Lecun] look a book,;\n[Lecun]work in the company.","","bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn,amputation, disconnected limbs","No_style",1389292712,"randomize",20,7,0.5,20,"none",0.8,0.5,5,false,"none",0.8,"0., 0.25, 0.4, 0.75;0.6, 0.25, 1., 0.75"]},{"id":16,"type":"Storydiffusion_Model_Loader","pos":{"0":388,"1":-177},"size":{"0":369.6249084472656,"1":442},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"model","type":"STORY_DICT","links":[24],"slot_index":0,"shape":3,"label":"model"}],"properties":{"Node name for S&R":"Storydiffusion_Model_Loader"},"widgets_values":["","sdxlYamersRealistic5_v5SX.safetensors","none","none","none",0.8,"best quality","Euler","txt2img",2,0.5,0.5,768,768,"v1",false]}],"links":[[24,16,0,19,0,"STORY_DICT"],[26,19,0,12,0,"IMAGE"]],"groups":[],"config":{},"extra":{"ds":{"scale":1.1167815779424883,"offset":[-341.40946511841486,263.2960533058556]}},"version":0.4}



## Additional Context
(Please add any additional context or steps to reproduce the error here)
smthemex commented 2 months ago

OK,new update I try some “mps” codes,I haven't tested M1, so if you have any issues, please reply to me on this page