search

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
58.28k stars 6.19k forks source link

Allocation on device | torch.OutOfMemoryError | SamplerCustomAdvanced | Flux.1 #4936

Open Azrox01 opened 2 months ago

Azrox01 commented 2 months ago

Your question

it was working fine yesterday but now I am having this error... I don't know why, it's my first time using an image-generation model so I don't know what to do. it is working fine using an image from ComfyUI_examples as a workflow, it is occurring when I am using a workflow with LoRA or I don't know if something else is happening in the background, but it was working fine before even with the LoRA and now its giving Allocation on device error.

Logs

# ComfyUI Error Report
## Error Details
- **Node Type:** SamplerCustomAdvanced
- **Exception Type:** torch.OutOfMemoryError
- **Exception Message:** Allocation on device 
## Stack Trace

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy_extras\nodes_custom_sampler.py", line 612, in sample
    samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 706, in sample
    self.inner_model, self.conds, self.loaded_models = comfy.sampler_helpers.prepare_sampling(self.model_patcher, noise.shape, self.conds)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\sampler_helpers.py", line 66, in prepare_sampling
    comfy.model_management.load_models_gpu([model] + models, memory_required=memory_required, minimum_memory_required=minimum_memory_required)

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 545, in load_models_gpu
    cur_loaded_model = loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 327, in model_load
    raise e

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 323, in model_load
    self.real_model = self.model.patch_model(device_to=patch_model_to, lowvram_model_memory=lowvram_model_memory, load_weights=load_weights, force_patch_weights=force_patch_weights)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 427, in patch_model
    self.load(device_to, lowvram_model_memory=lowvram_model_memory, force_patch_weights=force_patch_weights, full_load=full_load)

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 393, in load
    self.patch_weight_to_device(weight_key, device_to=device_to)

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 320, in patch_weight_to_device
    temp_weight = comfy.model_management.cast_to_device(weight, device_to, torch.float32, copy=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 857, in cast_to_device
    return tensor.to(device, copy=copy, non_blocking=non_blocking).to(dtype, non_blocking=non_blocking)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

System Information

Logs

2024-09-16 01:15:35,690 - root - INFO - Total VRAM 24576 MB, total RAM 16305 MB
2024-09-16 01:15:35,691 - root - INFO - pytorch version: 2.4.1+cu124
2024-09-16 01:15:35,691 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-16 01:15:35,691 - root - INFO - Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
2024-09-16 01:15:36,433 - root - INFO - Using pytorch cross attention
2024-09-16 01:15:37,505 - root - INFO - [Prompt Server] web root: E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\web
2024-09-16 01:15:38,351 - root - INFO - 
Import times for custom nodes:
2024-09-16 01:15:38,352 - root - INFO -    0.0 seconds: E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py
2024-09-16 01:15:38,352 - root - INFO -    0.0 seconds: E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfy-image-saver
2024-09-16 01:15:38,352 - root - INFO -    0.1 seconds: E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Crystools
2024-09-16 01:15:38,352 - root - INFO -    0.2 seconds: E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\x-flux-comfyui
2024-09-16 01:15:38,352 - root - INFO -    0.3 seconds: E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
2024-09-16 01:15:38,352 - root - INFO - 
2024-09-16 01:15:38,357 - root - INFO - Starting server

2024-09-16 01:15:38,357 - root - INFO - To see the GUI go to: http://127.0.0.1:8188
2024-09-16 01:17:48,718 - root - INFO - got prompt
2024-09-16 01:17:48,720 - root - ERROR - Failed to validate prompt for output 9:
2024-09-16 01:17:48,720 - root - ERROR - * UNETLoader 12:
2024-09-16 01:17:48,720 - root - ERROR -   - Value not in list: unet_name: 'flux1-dev-fp8.safetensors' not in ['flux1-dev.safetensors', 'flux1-schnell.safetensors']
2024-09-16 01:17:48,721 - root - ERROR - Output will be ignored
2024-09-16 01:17:48,721 - root - WARNING - invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}
2024-09-16 01:18:08,877 - root - INFO - got prompt
2024-09-16 01:18:08,923 - root - INFO - Using pytorch attention in VAE
2024-09-16 01:18:08,925 - root - INFO - Using pytorch attention in VAE
2024-09-16 01:18:14,244 - root - INFO - model weight dtype torch.bfloat16, manual cast: None
2024-09-16 01:18:14,252 - root - INFO - model_type FLUX
2024-09-16 01:19:32,926 - root - INFO - Requested to load FluxClipModel_
2024-09-16 01:19:32,926 - root - INFO - Loading 1 new model
2024-09-16 01:19:32,936 - root - INFO - loaded completely 0.0 9319.23095703125 True
2024-09-16 01:19:44,434 - root - WARNING - clip missing: ['text_projection.weight']
2024-09-16 01:19:46,098 - root - INFO - Requested to load Flux
2024-09-16 01:19:46,098 - root - INFO - Loading 1 new model
2024-09-16 01:20:26,283 - root - ERROR - ERROR lora diffusion_model.double_blocks.15.img_mlp.0.weight Allocation on device 
2024-09-16 01:20:26,424 - root - ERROR - ERROR lora diffusion_model.double_blocks.14.txt_mlp.0.weight Allocation on device 
2024-09-16 01:20:26,576 - root - ERROR - ERROR lora diffusion_model.double_blocks.14.img_mlp.0.weight Allocation on device 
2024-09-16 01:20:26,858 - root - ERROR - !!! Exception during processing !!! Allocation on device 
2024-09-16 01:20:26,905 - root - ERROR - Traceback (most recent call last):
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy_extras\nodes_custom_sampler.py", line 612, in sample
    samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 706, in sample
    self.inner_model, self.conds, self.loaded_models = comfy.sampler_helpers.prepare_sampling(self.model_patcher, noise.shape, self.conds)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\sampler_helpers.py", line 66, in prepare_sampling
    comfy.model_management.load_models_gpu([model] + models, memory_required=memory_required, minimum_memory_required=minimum_memory_required)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 545, in load_models_gpu
    cur_loaded_model = loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 327, in model_load
    raise e
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 323, in model_load
    self.real_model = self.model.patch_model(device_to=patch_model_to, lowvram_model_memory=lowvram_model_memory, load_weights=load_weights, force_patch_weights=force_patch_weights)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 427, in patch_model
    self.load(device_to, lowvram_model_memory=lowvram_model_memory, force_patch_weights=force_patch_weights, full_load=full_load)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 393, in load
    self.patch_weight_to_device(weight_key, device_to=device_to)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 320, in patch_weight_to_device
    temp_weight = comfy.model_management.cast_to_device(weight, device_to, torch.float32, copy=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 857, in cast_to_device
    return tensor.to(device, copy=copy, non_blocking=non_blocking).to(dtype, non_blocking=non_blocking)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: Allocation on device 

2024-09-16 01:20:26,910 - root - ERROR - Got an OOM, unloading all loaded models.
2024-09-16 01:20:27,032 - root - INFO - Prompt executed in 138.15 seconds
2024-09-16 01:37:11,648 - root - INFO - got prompt
2024-09-16 01:37:11,717 - root - INFO - Requested to load FluxClipModel_
2024-09-16 01:37:11,717 - root - INFO - Loading 1 new model
2024-09-16 01:37:29,424 - root - INFO - loaded completely 0.0 9319.23095703125 True
2024-09-16 01:37:29,744 - root - INFO - Requested to load Flux
2024-09-16 01:37:29,744 - root - INFO - Loading 1 new model
2024-09-16 01:38:12,806 - root - ERROR - ERROR lora diffusion_model.double_blocks.17.img_mlp.0.weight Allocation on device 
2024-09-16 01:38:12,956 - root - ERROR - ERROR lora diffusion_model.double_blocks.16.txt_mlp.0.weight Allocation on device 
2024-09-16 01:38:13,268 - root - ERROR - ERROR lora diffusion_model.double_blocks.15.txt_mlp.0.weight Allocation on device 
2024-09-16 01:38:13,404 - root - ERROR - ERROR lora diffusion_model.double_blocks.15.img_mlp.0.weight Allocation on device 
2024-09-16 01:38:13,555 - root - ERROR - ERROR lora diffusion_model.double_blocks.14.txt_mlp.0.weight Allocation on device 
2024-09-16 01:38:13,839 - root - ERROR - !!! Exception during processing !!! Allocation on device 
2024-09-16 01:38:13,873 - root - ERROR - Traceback (most recent call last):
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy_extras\nodes_custom_sampler.py", line 612, in sample
    samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 706, in sample
    self.inner_model, self.conds, self.loaded_models = comfy.sampler_helpers.prepare_sampling(self.model_patcher, noise.shape, self.conds)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\sampler_helpers.py", line 66, in prepare_sampling
    comfy.model_management.load_models_gpu([model] + models, memory_required=memory_required, minimum_memory_required=minimum_memory_required)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 545, in load_models_gpu
    cur_loaded_model = loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 327, in model_load
    raise e
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 323, in model_load
    self.real_model = self.model.patch_model(device_to=patch_model_to, lowvram_model_memory=lowvram_model_memory, load_weights=load_weights, force_patch_weights=force_patch_weights)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 427, in patch_model
    self.load(device_to, lowvram_model_memory=lowvram_model_memory, force_patch_weights=force_patch_weights, full_load=full_load)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 393, in load
    self.patch_weight_to_device(weight_key, device_to=device_to)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 320, in patch_weight_to_device
    temp_weight = comfy.model_management.cast_to_device(weight, device_to, torch.float32, copy=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 857, in cast_to_device
    return tensor.to(device, copy=copy, non_blocking=non_blocking).to(dtype, non_blocking=non_blocking)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: Allocation on device 

2024-09-16 01:38:13,877 - root - ERROR - Got an OOM, unloading all loaded models.
2024-09-16 01:38:14,002 - root - INFO - Prompt executed in 62.35 seconds
2024-09-16 01:40:06,294 - root - INFO - got prompt
2024-09-16 01:40:06,546 - root - INFO - Requested to load FluxClipModel_
2024-09-16 01:40:06,546 - root - INFO - Loading 1 new model
2024-09-16 01:40:25,405 - root - INFO - loaded completely 0.0 9319.23095703125 True
2024-09-16 01:40:25,736 - root - INFO - Requested to load Flux
2024-09-16 01:40:25,737 - root - INFO - Loading 1 new model
2024-09-16 01:41:07,730 - root - ERROR - ERROR lora diffusion_model.double_blocks.16.txt_mlp.0.weight Allocation on device 
2024-09-16 01:41:07,866 - root - ERROR - ERROR lora diffusion_model.double_blocks.16.img_mlp.0.weight Allocation on device 
2024-09-16 01:41:08,000 - root - ERROR - ERROR lora diffusion_model.double_blocks.15.txt_mlp.0.weight Allocation on device 
2024-09-16 01:41:08,215 - root - ERROR - !!! Exception during processing !!! Allocation on device 
2024-09-16 01:41:08,237 - root - ERROR - Traceback (most recent call last):
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy_extras\nodes_custom_sampler.py", line 612, in sample
    samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 706, in sample
    self.inner_model, self.conds, self.loaded_models = comfy.sampler_helpers.prepare_sampling(self.model_patcher, noise.shape, self.conds)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\sampler_helpers.py", line 66, in prepare_sampling
    comfy.model_management.load_models_gpu([model] + models, memory_required=memory_required, minimum_memory_required=minimum_memory_required)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 545, in load_models_gpu
    cur_loaded_model = loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 327, in model_load
    raise e
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 323, in model_load
    self.real_model = self.model.patch_model(device_to=patch_model_to, lowvram_model_memory=lowvram_model_memory, load_weights=load_weights, force_patch_weights=force_patch_weights)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 427, in patch_model
    self.load(device_to, lowvram_model_memory=lowvram_model_memory, force_patch_weights=force_patch_weights, full_load=full_load)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 393, in load
    self.patch_weight_to_device(weight_key, device_to=device_to)
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_patcher.py", line 320, in patch_weight_to_device
    temp_weight = comfy.model_management.cast_to_device(weight, device_to, torch.float32, copy=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI\ComfyUI_windows_portable\ComfyUI\comfy\model_management.py", line 857, in cast_to_device
    return tensor.to(device, copy=copy, non_blocking=non_blocking).to(dtype, non_blocking=non_blocking)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: Allocation on device 

2024-09-16 01:41:08,240 - root - ERROR - Got an OOM, unloading all loaded models.
2024-09-16 01:41:08,368 - root - INFO - Prompt executed in 62.07 seconds

Attached Workflow

Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":72,"last_link_id":108,"nodes":[{"id":6,"type":"CLIPTextEncode","pos":{"0":424.71875,"1":618.052001953125},"size":{"0":210,"1":54},"flags":{"collapsed":false},"order":12,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":108},{"name":"text","type":"STRING","link":47,"slot_index":1,"widget":{"name":"text"}}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[86],"slot_index":0}],"properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":[""]},{"id":25,"type":"RandomNoise","pos":{"0":424,"1":236},"size":{"0":327.1990661621094,"1":94.58134460449219},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"NOISE","type":"NOISE","links":[37],"shape":3}],"properties":{"Node name for S&R":"RandomNoise"},"widgets_values":[655720647218432,"randomize"]},{"id":5,"type":"EmptyLatentImage","pos":{"0":422,"1":101},"size":{"0":330.5548400878906,"1":78},"flags":{},"order":10,"mode":0,"inputs":[{"name":"width","type":"INT","link":101,"widget":{"name":"width"}},{"name":"height","type":"INT","link":103,"widget":{"name":"height"}}],"outputs":[{"name":"LATENT","type":"LATENT","links":[23],"slot_index":0}],"properties":{"Node name for S&R":"EmptyLatentImage"},"widgets_values":[832,1216,1]},{"id":16,"type":"KSamplerSelect","pos":{"0":809,"1":261},"size":{"0":268.2277526855469,"1":58},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"SAMPLER","type":"SAMPLER","links":[19],"shape":3}],"properties":{"Node name for S&R":"KSamplerSelect"},"widgets_values":["euler"]},{"id":22,"type":"BasicGuider","pos":{"0":893.71875,"1":612.052001953125},"size":{"0":196.9998779296875,"1":62.66668701171875},"flags":{"collapsed":false},"order":16,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":94,"slot_index":0},{"name":"conditioning","type":"CONDITIONING","link":87,"slot_index":1}],"outputs":[{"name":"GUIDER","type":"GUIDER","links":[30],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"BasicGuider"}},{"id":52,"type":"Note","pos":{"0":1148.09375,"1":611.84375},"size":{"0":346.2236022949219,"1":58},"flags":{},"order":2,"mode":0,"inputs":[],"outputs":[],"properties":{"text":""},"widgets_values":["To see the preview, update your ComfyUI and go into the Manager menu. Set \"Preview Method\" to \"Auto\""],"color":"#432","bgcolor":"#653"},{"id":13,"type":"SamplerCustomAdvanced","pos":{"0":1143.252685546875,"1":89.17115783691406},"size":{"0":352.4039611816406,"1":463.3393859863281},"flags":{},"order":17,"mode":0,"inputs":[{"name":"noise","type":"NOISE","link":37,"slot_index":0},{"name":"guider","type":"GUIDER","link":30,"slot_index":1},{"name":"sampler","type":"SAMPLER","link":19,"slot_index":2},{"name":"sigmas","type":"SIGMAS","link":20,"slot_index":3},{"name":"latent_image","type":"LATENT","link":23,"slot_index":4}],"outputs":[{"name":"output","type":"LATENT","links":[24],"slot_index":0,"shape":3},{"name":"denoised_output","type":"LATENT","links":null,"shape":3}],"properties":{"Node name for S&R":"SamplerCustomAdvanced"}},{"id":53,"type":"Note","pos":{"0":381,"1":738},"size":{"0":548.0299072265625,"1":283.5565490722656},"flags":{},"order":3,"mode":0,"inputs":[],"outputs":[],"properties":{"text":""},"widgets_values":["The checkpoint goes in ComfyUI/models/unet (not checkpoints)\nDownload the original weights here:\nhttps://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/flux1-dev.sft\n\nDownload the fp8 version for <24gb vram systems:\nhttps://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors\n\nText encoders go in ComfyUI/models/clip:\nhttps://huggingface.co/comfyanonymous/flux_text_encoders/tree/main\n\nVAE (ae.sft) goes in ComfyUI/models/vae:\nhttps://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.sft\n\nDownload the fp8 t5xxl for degraded quality but less RAM use\nLaunch ComfyUI with \"--lowvram\" arg (in the .bat file) to offload text encoder to CPU.\n\nI can confirm this runs on:\n- RTX 3090 (24gb) 1.29s/it\n- RTX 4070 (12gb) 85s/it\nBoth running the fp8 quantized version. The 4070 is very slow though."],"color":"#432","bgcolor":"#653"},{"id":8,"type":"VAEDecode","pos":{"0":1546,"1":88},"size":{"0":210,"1":46},"flags":{},"order":18,"mode":0,"inputs":[{"name":"samples","type":"LATENT","link":24},{"name":"vae","type":"VAE","link":12}],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[9],"slot_index":0}],"properties":{"Node name for S&R":"VAEDecode"}},{"id":9,"type":"SaveImage","pos":{"0":1784,"1":88},"size":{"0":399.1837463378906,"1":508.5245666503906},"flags":{},"order":19,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":9}],"outputs":[],"properties":{"Node name for S&R":"SaveImage"},"widgets_values":["MarkuryFLUX"]},{"id":61,"type":"ModelSamplingFlux","pos":{"0":754,"1":383},"size":{"0":321.8402404785156,"1":122},"flags":{},"order":13,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":106},{"name":"width","type":"INT","link":102,"widget":{"name":"width"}},{"name":"height","type":"INT","link":104,"widget":{"name":"height"}}],"outputs":[{"name":"MODEL","type":"MODEL","links":[93,94],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"ModelSamplingFlux"},"widgets_values":[1.15,0.5,1024,1024]},{"id":70,"type":"Int Literal","pos":{"0":25,"1":495},"size":{"0":315,"1":58},"flags":{},"order":4,"mode":0,"inputs":[],"outputs":[{"name":"INT","type":"INT","links":[101,102],"slot_index":0,"shape":3}],"title":"Width","properties":{"Node name for S&R":"Int Literal"},"widgets_values":[832]},{"id":71,"type":"Int Literal","pos":{"0":28,"1":610},"size":{"0":315,"1":58},"flags":{},"order":5,"mode":0,"inputs":[],"outputs":[{"name":"INT","type":"INT","links":[103,104],"slot_index":0,"shape":3}],"title":"Height","properties":{"Node name for S&R":"Int Literal"},"widgets_values":[1216]},{"id":60,"type":"FluxGuidance","pos":{"0":659,"1":614},"size":{"0":211.60000610351562,"1":58},"flags":{},"order":14,"mode":0,"inputs":[{"name":"conditioning","type":"CONDITIONING","link":86}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[87],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"FluxGuidance"},"widgets_values":[3.5],"color":"#323","bgcolor":"#535"},{"id":11,"type":"DualCLIPLoader","pos":{"0":22,"1":214},"size":{"0":315,"1":106},"flags":{},"order":6,"mode":0,"inputs":[],"outputs":[{"name":"CLIP","type":"CLIP","links":[108],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"DualCLIPLoader"},"widgets_values":["t5xxl_fp16.safetensors","clip_l.safetensors","flux"]},{"id":10,"type":"VAELoader","pos":{"0":26,"1":379},"size":{"0":315,"1":58},"flags":{},"order":7,"mode":0,"inputs":[],"outputs":[{"name":"VAE","type":"VAE","links":[12],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"VAELoader"},"widgets_values":["ae.safetensors"]},{"id":17,"type":"BasicScheduler","pos":{"0":797,"1":94},"size":{"0":281.2428283691406,"1":106},"flags":{},"order":15,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":93,"slot_index":0}],"outputs":[{"name":"SIGMAS","type":"SIGMAS","links":[20],"shape":3}],"properties":{"Node name for S&R":"BasicScheduler"},"widgets_values":["simple",20,1]},{"id":12,"type":"UNETLoader","pos":{"0":18,"1":84},"size":{"0":315,"1":82},"flags":{},"order":8,"mode":0,"inputs":[],"outputs":[{"name":"MODEL","type":"MODEL","links":[107],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"UNETLoader"},"widgets_values":["flux1-dev.safetensors","default"]},{"id":72,"type":"LoraLoaderModelOnly","pos":{"0":419,"1":403},"size":{"0":315,"1":82},"flags":{},"order":11,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":107}],"outputs":[{"name":"MODEL","type":"MODEL","links":[106],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"LoraLoaderModelOnly"},"widgets_values":["nami_v1.safetensors",1]},{"id":28,"type":"String Literal","pos":{"0":26.5353946685791,"1":790.639892578125},"size":{"0":317.8795471191406,"1":202.01535034179688},"flags":{},"order":9,"mode":0,"inputs":[],"outputs":[{"name":"STRING","type":"STRING","links":[47],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"String Literal"},"widgets_values":["Nami from one piece"]}],"links":[[9,8,0,9,0,"IMAGE"],[12,10,0,8,1,"VAE"],[19,16,0,13,2,"SAMPLER"],[20,17,0,13,3,"SIGMAS"],[23,5,0,13,4,"LATENT"],[24,13,0,8,0,"LATENT"],[30,22,0,13,1,"GUIDER"],[37,25,0,13,0,"NOISE"],[47,28,0,6,1,"STRING"],[86,6,0,60,0,"CONDITIONING"],[87,60,0,22,1,"CONDITIONING"],[93,61,0,17,0,"MODEL"],[94,61,0,22,0,"MODEL"],[101,70,0,5,0,"INT"],[102,70,0,61,1,"INT"],[103,71,0,5,1,"INT"],[104,71,0,61,2,"INT"],[106,72,0,61,0,"MODEL"],[107,12,0,72,0,"MODEL"],[108,11,0,6,0,"CLIP"]],"groups":[{"title":"Load FLUX.1","bounding":[1,2,369,693],"color":"#3f789e","font_size":24,"flags":{}},{"title":"Set Parameters","bounding":[379,0,733,526],"color":"#3f789e","font_size":24,"flags":{}},{"title":"FLUX Prompt","bounding":[1,704,368,318],"color":"#3f789e","font_size":24,"flags":{}},{"title":"Conditioning","bounding":[379,535,732,159],"color":"#3f789e","font_size":24,"flags":{}},{"title":"1st Pass","bounding":[1119,0,402,693],"color":"#3f789e","font_size":24,"flags":{}}],"config":{},"extra":{"ds":{"scale":0.9090909090909091,"offset":[239.11152152765774,97.91972990135856]}},"version":0.4}

Additional Context

(Please add any additional context or steps to reproduce the error here)

unexpectedly commented 2 months ago

Similar experience - only when using xlabs flux nodes. (running on Colab Pro)

github-actions[bot] commented 1 month ago

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.

txhno commented 1 month ago

I still have this issue.

walterFeng commented 3 weeks ago

If using flux, try not to select default for weight_dtype of the LoadDiffusionModel node. But I also encountered this bug when I did not apply flux.