WASasquatch / was-node-suite-comfyui

An extensive node suite for ComfyUI with over 210 new nodes
MIT License
1.15k stars 170 forks source link

Convert images loaded by WAS_Load_Image_Batch to RGB format #118

Closed jamesWalker55 closed 1 year ago

jamesWalker55 commented 1 year ago

When using an RGBA image as the input to a "Apply ControlNet" node, the KSampler node raises an error:

!!! Exception during processing !!!
Traceback (most recent call last):
  File "D:\stable-diffusion\ComfyUI\execution.py", line 145, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "D:\stable-diffusion\ComfyUI\execution.py", line 75, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "D:\stable-diffusion\ComfyUI\execution.py", line 68, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "D:\stable-diffusion\ComfyUI\nodes.py", line 1080, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
  File "D:\stable-diffusion\ComfyUI\nodes.py", line 1050, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
  File "D:\stable-diffusion\ComfyUI\comfy\sample.py", line 88, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "D:\stable-diffusion\ComfyUI\comfy\samplers.py", line 670, in sample
    samples = getattr(k_diffusion_sampling, "sample_{}".format(self.sampler))(self.model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\k_diffusion\sampling.py", line 605, in sample_dpmpp_2m
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\samplers.py", line 307, in forward
    out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, cond_concat=cond_concat, model_options=model_options, seed=seed)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\k_diffusion\external.py", line 114, in forward
    eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\k_diffusion\external.py", line 140, in get_eps
    return self.inner_model.apply_model(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\samplers.py", line 295, in apply_model
    out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
  File "D:\stable-diffusion\ComfyUI\comfy\samplers.py", line 273, in sampling_function
    cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
  File "D:\stable-diffusion\ComfyUI\comfy\samplers.py", line 231, in calc_cond_uncond_batch
    c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
  File "D:\stable-diffusion\ComfyUI\comfy\sd.py", line 685, in get_control
    control = self.control_model(x=x_noisy, hint=self.cond_hint, timesteps=t, context=context, y=y)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\cldm\cldm.py", line 292, in forward
    guided_hint = self.input_hint_block(hint, emb, context)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 88, in forward
    x = layer(x)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "D:\stable-diffusion\ComfyUI\venv\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [16, 3, 3, 3], expected input[1, 4, 1776, 696] to have 3 channels, but got 4 channels instead

This occurs when using WAS_Load_Image_Batch as the image loader. It loads images in their original format, which may lead to different number of channels if the image isn't in RGB format. The tensor will then cause the above error in KSampler.

This commit adds .convert('RGB') in the WAS_Load_Image_Batch class to ensure the loaded images are in RGB format.

WASasquatch commented 1 year ago

You also couldn't then load and manipulate alpha PNG images as raster masks, or overlay elements. There is a nodes to extract and compile RGB. The a simple RGBA to RGB node would be more convenient.

jamesWalker55 commented 1 year ago

What's the name of the node to convert RGBA to RGB? I couldn't find it in the nodes list.

WASasquatch commented 1 year ago

What's the name of the node to convert RGBA to RGB? I couldn't find it in the nodes list.

There is currently RGB channel separation and compilation. I'll add a node for RGBA to RGB, but I think retaining RGBA is important at load time as you may not be immediately diffusing it.

WASasquatch commented 1 year ago

I've added a RGBA to RGB node