search

smthemex / ComfyUI_StoryDiffusion

You can using StoryDiffusion in ComfyUI
Apache License 2.0
241 stars 195 forks source link

- **Exception Message:** Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) #95

Closed mitchellfox closed 1 month ago

mitchellfox commented 1 month ago

ComfyUI Error Report

Error Details

## System Information
- **ComfyUI Version:** v0.2.2-83-gb4626ab
- **Arguments:** main.py --port 4001 --listen 0.0.0.0 --gpu-only
- **OS:** posix
- **Python Version:** 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
- **Embedded Python:** false
- **PyTorch Version:** 2.4.1+cu124
## Devices

- **Name:** cuda:0 NVIDIA A100-SXM4-40GB : cudaMallocAsync
  - **Type:** cuda
  - **VRAM Total:** 42298834944
  - **VRAM Free:** 34459687144
  - **Torch VRAM Total:** 7314866176
  - **Torch VRAM Free:** 47585512

## Logs

2024-10-01 15:21:03,147 - root - INFO - Total VRAM 40339 MB, total RAM 221442 MB 2024-10-01 15:21:03,147 - root - INFO - pytorch version: 2.4.1+cu124 2024-10-01 15:21:03,148 - root - INFO - Set vram state to: HIGH_VRAM 2024-10-01 15:21:03,148 - root - INFO - Device: cuda:0 NVIDIA A100-SXM4-40GB : cudaMallocAsync 2024-10-01 15:21:04,020 - root - INFO - Using pytorch cross attention 2024-10-01 15:21:04,881 - root - INFO - [Prompt Server] web root: /home/ubuntu/ComfyUI/web 2024-10-01 15:21:07,051 - root - INFO - -------------- 2024-10-01 15:21:07,051 - root - INFO -  ### Mixlab Nodes: Loaded 2024-10-01 15:21:07,051 - root - INFO - ChatGPT.available True 2024-10-01 15:21:07,052 - root - INFO - editmask.available True 2024-10-01 15:21:07,247 - root - INFO - ClipInterrogator.available True 2024-10-01 15:21:07,508 - root - INFO - PromptGenerate.available True 2024-10-01 15:21:07,508 - root - INFO - ChinesePrompt.available True 2024-10-01 15:21:07,508 - root - INFO - RembgNode.available True 2024-10-01 15:21:07,894 - root - INFO - TripoSR.available 2024-10-01 15:21:07,894 - root - INFO - MiniCPMNode.available 2024-10-01 15:21:07,985 - root - INFO - Scenedetect.available 2024-10-01 15:21:07,987 - root - INFO - FishSpeech.available False 2024-10-01 15:21:07,987 - root - INFO -  --------------  2024-10-01 15:21:09,425 - root - INFO - Total VRAM 40339 MB, total RAM 221442 MB 2024-10-01 15:21:09,426 - root - INFO - pytorch version: 2.4.1+cu124 2024-10-01 15:21:09,426 - root - INFO - Set vram state to: HIGH_VRAM 2024-10-01 15:21:09,426 - root - INFO - Device: cuda:0 NVIDIA A100-SXM4-40GB : cudaMallocAsync 2024-10-01 15:21:09,432 - root - INFO - Import times for custom nodes: 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/websocket_image_save.py 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/cg-use-everywhere 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/A8R8_ComfyUI_nodes 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/masquerade-nodes-comfyui 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/comfyui_controlnet_aux 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/comfyui_segment_anything 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI_essentials 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-Allor 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/rgthree-comfy 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-KJNodes 2024-10-01 15:21:09,432 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes 2024-10-01 15:21:09,433 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved 2024-10-01 15:21:09,433 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/efficiency-nodes-comfyui 2024-10-01 15:21:09,433 - root - INFO - 0.0 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-Manager 2024-10-01 15:21:09,433 - root - INFO - 0.1 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-tbox 2024-10-01 15:21:09,433 - root - INFO - 0.2 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion 2024-10-01 15:21:09,433 - root - INFO - 0.3 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI_InstantID 2024-10-01 15:21:09,433 - root - INFO - 0.3 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-Impact-Pack 2024-10-01 15:21:09,433 - root - INFO - 0.4 seconds: /home/ubuntu/ComfyUI/custom_nodes/comfyui-reactor-node 2024-10-01 15:21:09,433 - root - INFO - 0.5 seconds: /home/ubuntu/ComfyUI/custom_nodes/ComfyUI-SAM2 2024-10-01 15:21:09,433 - root - INFO - 2.2 seconds: /home/ubuntu/ComfyUI/custom_nodes/comfyui-mixlab-nodes 2024-10-01 15:21:09,684 - root - INFO - 2024-10-01 15:21:09,699 - root - INFO - Starting server

2024-10-01 15:21:09,699 - root - INFO - To see the GUI go to: http://0.0.0.0:4001 2024-10-01 15:21:16,232 - root - INFO - got prompt 2024-10-01 15:21:16,445 - root - INFO - Requested to load FluxClipModel_ 2024-10-01 15:21:16,445 - root - INFO - Loading 1 new model 2024-10-01 15:21:16,457 - root - INFO - loaded completely 0.0 4777.53759765625 True 2024-10-01 15:21:17,199 - root - WARNING - clip missing: ['text_projection.weight'] 2024-10-01 15:21:17,542 - root - INFO - Process using 1 roles,mode is img2img.... 2024-10-01 15:21:17,542 - root - INFO - total_vram is 40339.3125,aggressive_offload is False,offload is False 2024-10-01 15:21:17,542 - root - INFO - start flux-pulid processing... 2024-10-01 15:23:20,346 - root - INFO - loading weight_dtype is None 2024-10-01 15:23:29,661 - root - INFO - Loaded EVA02-CLIP-L-14-336 model config. 2024-10-01 15:23:29,674 - root - INFO - Shape of rope freq: torch.Size([576, 64]) 2024-10-01 15:23:36,617 - root - INFO - Loading pretrained EVA02-CLIP-L-14-336 weights (/home/ubuntu/ComfyUI/models/clip_vision/EVA02_CLIP_L_336_psz14_s6B.pt). 2024-10-01 15:23:37,281 - root - INFO - incompatible_keys.missing_keys: ['visual.rope.freqs_cos', 'visual.rope.freqs_sin', 'visual.blocks.0.attn.rope.freqs_cos', 'visual.blocks.0.attn.rope.freqs_sin', 'visual.blocks.1.attn.rope.freqs_cos', 'visual.blocks.1.attn.rope.freqs_sin', 'visual.blocks.2.attn.rope.freqs_cos', 'visual.blocks.2.attn.rope.freqs_sin', 'visual.blocks.3.attn.rope.freqs_cos', 'visual.blocks.3.attn.rope.freqs_sin', 'visual.blocks.4.attn.rope.freqs_cos', 'visual.blocks.4.attn.rope.freqs_sin', 'visual.blocks.5.attn.rope.freqs_cos', 'visual.blocks.5.attn.rope.freqs_sin', 'visual.blocks.6.attn.rope.freqs_cos', 'visual.blocks.6.attn.rope.freqs_sin', 'visual.blocks.7.attn.rope.freqs_cos', 'visual.blocks.7.attn.rope.freqs_sin', 'visual.blocks.8.attn.rope.freqs_cos', 'visual.blocks.8.attn.rope.freqs_sin', 'visual.blocks.9.attn.rope.freqs_cos', 'visual.blocks.9.attn.rope.freqs_sin', 'visual.blocks.10.attn.rope.freqs_cos', 'visual.blocks.10.attn.rope.freqs_sin', 'visual.blocks.11.attn.rope.freqs_cos', 'visual.blocks.11.attn.rope.freqs_sin', 'visual.blocks.12.attn.rope.freqs_cos', 'visual.blocks.12.attn.rope.freqs_sin', 'visual.blocks.13.attn.rope.freqs_cos', 'visual.blocks.13.attn.rope.freqs_sin', 'visual.blocks.14.attn.rope.freqs_cos', 'visual.blocks.14.attn.rope.freqs_sin', 'visual.blocks.15.attn.rope.freqs_cos', 'visual.blocks.15.attn.rope.freqs_sin', 'visual.blocks.16.attn.rope.freqs_cos', 'visual.blocks.16.attn.rope.freqs_sin', 'visual.blocks.17.attn.rope.freqs_cos', 'visual.blocks.17.attn.rope.freqs_sin', 'visual.blocks.18.attn.rope.freqs_cos', 'visual.blocks.18.attn.rope.freqs_sin', 'visual.blocks.19.attn.rope.freqs_cos', 'visual.blocks.19.attn.rope.freqs_sin', 'visual.blocks.20.attn.rope.freqs_cos', 'visual.blocks.20.attn.rope.freqs_sin', 'visual.blocks.21.attn.rope.freqs_cos', 'visual.blocks.21.attn.rope.freqs_sin', 'visual.blocks.22.attn.rope.freqs_cos', 'visual.blocks.22.attn.rope.freqs_sin', 'visual.blocks.23.attn.rope.freqs_cos', 'visual.blocks.23.attn.rope.freqs_sin'] 2024-10-01 15:23:42,323 - root - ERROR - !!! Exception during processing !!! Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) 2024-10-01 15:23:42,325 - root - ERROR - Traceback (most recent call last): File "/home/ubuntu/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/home/ubuntu/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 512, in story_sampler for value in gen: File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/model_loader_utils.py", line 1722, in process_generation id_images = pipe.generate_image( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/app_flux.py", line 141, in generate_image x = denoise( File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/sampling.py", line 135, in denoise pred = model( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/model.py", line 104, in forward img = self.img_in(img) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward return F.linear(input, self.weight, self.bias) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

2024-10-01 15:23:42,325 - root - INFO - Prompt executed in 146.09 seconds 2024-10-01 15:26:27,779 - root - INFO - got prompt 2024-10-01 15:26:28,230 - root - ERROR - !!! Exception during processing !!! Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) 2024-10-01 15:26:28,230 - root - ERROR - Traceback (most recent call last): File "/home/ubuntu/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/home/ubuntu/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 512, in story_sampler for value in gen: File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/model_loader_utils.py", line 1722, in process_generation id_images = pipe.generate_image( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/app_flux.py", line 141, in generate_image x = denoise( File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/sampling.py", line 135, in denoise pred = model( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/model.py", line 104, in forward img = self.img_in(img) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward return F.linear(input, self.weight, self.bias) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

2024-10-01 15:26:28,230 - root - INFO - Prompt executed in 0.45 seconds 2024-10-01 15:26:45,722 - root - INFO - got prompt 2024-10-01 15:26:46,168 - root - ERROR - !!! Exception during processing !!! Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) 2024-10-01 15:26:46,168 - root - ERROR - Traceback (most recent call last): File "/home/ubuntu/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/home/ubuntu/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 512, in story_sampler for value in gen: File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/model_loader_utils.py", line 1722, in process_generation id_images = pipe.generate_image( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/app_flux.py", line 141, in generate_image x = denoise( File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/sampling.py", line 135, in denoise pred = model( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/model.py", line 104, in forward img = self.img_in(img) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward return F.linear(input, self.weight, self.bias) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

2024-10-01 15:26:46,169 - root - INFO - Prompt executed in 0.44 seconds 2024-10-01 15:29:41,875 - root - INFO - got prompt 2024-10-01 15:29:42,314 - root - ERROR - !!! Exception during processing !!! Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) 2024-10-01 15:29:42,314 - root - ERROR - Traceback (most recent call last): File "/home/ubuntu/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/home/ubuntu/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/home/ubuntu/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/Storydiffusion_node.py", line 512, in story_sampler for value in gen: File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/model_loader_utils.py", line 1722, in process_generation id_images = pipe.generate_image( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, *kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/app_flux.py", line 141, in generate_image x = denoise( File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/sampling.py", line 135, in denoise pred = model( File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, kwargs) File "/home/ubuntu/ComfyUI/custom_nodes/ComfyUI_StoryDiffusion/PuLID/flux/model.py", line 104, in forward img = self.img_in(img) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) File "/home/ubuntu/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward return F.linear(input, self.weight, self.bias) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

2024-10-01 15:29:42,315 - root - INFO - Prompt executed in 0.44 seconds

## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":29,"last_link_id":46,"nodes":[{"id":19,"type":"LoadImage","pos":{"0":-109,"1":-430},"size":{"0":315,"1":314},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[43],"slot_index":0,"shape":3,"label":"IMAGE"},{"name":"MASK","type":"MASK","links":null,"shape":3,"label":"MASK"}],"properties":{"Node name for S&R":"LoadImage"},"widgets_values":["ComfyUI_temp_sudce00029.png","image"]},{"id":13,"type":"DualCLIPLoader","pos":{"0":-122,"1":-669},"size":{"0":315,"1":106},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"CLIP","type":"CLIP","links":[42],"slot_index":0,"shape":3,"label":"CLIP"}],"properties":{"Node name for S&R":"DualCLIPLoader"},"widgets_values":["t5xxl_fp8_e4m3fn.safetensors","clip_l.safetensors","flux"]},{"id":28,"type":"SaveImage","pos":{"0":1423.8323974609375,"1":-482.6542663574219},"size":{"0":315,"1":58},"flags":{},"order":4,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":46}],"outputs":[],"properties":{},"widgets_values":["ComfyUI"]},{"id":29,"type":"Storydiffusion_Sampler","pos":{"0":776,"1":-671},"size":{"0":402.78570556640625,"1":620.1786499023438},"flags":{},"order":3,"mode":0,"inputs":[{"name":"model","type":"STORY_DICT","link":45}],"outputs":[{"name":"image","type":"IMAGE","links":[46],"shape":3},{"name":"prompt_array","type":"STRING","links":null,"shape":3}],"properties":{"Node name for S&R":"Storydiffusion_Sampler"},"widgets_values":["[Taylor] wake up in the bed ;","bad anatomy, bad hands, missing fingers, extra fingers, three hands, three legs, bad arms, missing legs, missing arms, poorly drawn face, bad face, fused face, cloned face, three crus, fused feet, fused thigh, extra crus, ugly fingers, horn,amputation, disconnected limbs","No_style",1331813755,"randomize",20,7,1,20,3.5,0.5,5,false,0.8,"0., 0.25, 0.4, 0.75;0.6, 0.25, 1., 0.75"]},{"id":22,"type":"Storydiffusion_Model_Loader","pos":{"0":332,"1":-664},"size":{"0":415.4856872558594,"1":579.878662109375},"flags":{},"order":2,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":43,"label":"image"},{"name":"control_image","type":"IMAGE","link":null,"label":"control_image"},{"name":"model","type":"MODEL","link":null,"label":"model"},{"name":"clip","type":"CLIP","link":42,"label":"clip"},{"name":"vae","type":"VAE","link":null,"label":"vae"}],"outputs":[{"name":"model","type":"STORY_DICT","links":[45],"slot_index":0,"shape":3,"label":"model"}],"properties":{"Node name for S&R":"Storydiffusion_Model_Loader"},"widgets_values":["[Taylor] a woman img, wearing a GreenShirt, green loose hair.","","flux1-dev-fp8-e4m3fn.safetensors","ae.safetensors","none","GreenShirt_v1.safetensors",0.8,"none","EVA02_CLIP_L_336_psz14_s6B.pt","best quality","euler","normal",0.5,0.5,768,768,"v1","pulid,fp8"]}],"links":[[42,13,0,22,3,"CLIP"],[43,19,0,22,0,"IMAGE"],[45,22,0,29,0,"STORY_DICT"],[46,29,0,28,0,"IMAGE"]],"groups":[],"config":{},"extra":{"ds":{"scale":0.9646149645000006,"offset":[722.2557035993786,965.9689112192642]}},"version":0.4}



## Additional Context
(Please add any additional context or steps to reproduce the error here)

## Additional Context
Flux, PulID, fp8. I'm using the latest version of StoryDiffusion. I've tried running comfy with the --gpu-only flag and I still get this same error. I've tried with and without my GreenShirt flux LoRA, same result.
smthemex commented 1 month ago

bf16 + offload run python app_flux.py --offload, the peak memory is under 30GB. you VRAM is 40,use bf16 weights,

mitchellfox commented 1 month ago

Thank you!