[Bug]: IC Light is broken on dev_upstream

[dan4ik94]

After the dora fixes (3a3611ada4a42c1281c92cb3f23cc91c790cdae2) the IC Light extension stopped working, it throws an error related to shape mismatch.

I'm using the dev_upstream_1111 branch

Attaching the full traceback

WARNING:root:WARNING SHAPE MISMATCH diffusion_model.input_blocks.0.0.weight WEIGHT NOT MERGED torch.Size([320, 8, 3, 3]) != torch.Size([320, 4, 3, 3])

Traceback (most recent call last):
  File "D:\dan4ik-forge\modules_forge\main_thread.py", line 37, in loop
    task.work()
  File "D:\dan4ik-forge\modules_forge\main_thread.py", line 26, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "D:\dan4ik-forge\modules\txt2img.py", line 110, in txt2img_function
    processed = processing.process_images(p)
  File "D:\dan4ik-forge\modules\processing.py", line 815, in process_images
    res = process_images_inner(p)
  File "D:\dan4ik-forge\modules\processing.py", line 988, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "D:\dan4ik-forge\modules\processing.py", line 1362, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "D:\dan4ik-forge\modules\sd_samplers_kdiffusion.py", line 261, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "D:\dan4ik-forge\modules\sd_samplers_common.py", line 274, in launch_sampling
    return func()
  File "D:\dan4ik-forge\modules\sd_samplers_kdiffusion.py", line 261, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\dan4ik-forge\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\dan4ik-forge\modules\sd_samplers_cfg_denoiser.py", line 369, in forward
    denoised = sampling_function(model, x, sigma, uncond_patched, cond_patched, cond_scale, model_options, seed)
  File "D:\dan4ik-forge\ldm_patched\modules\samplers.py", line 293, in sampling_function
    cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
  File "D:\dan4ik-forge\ldm_patched\modules\samplers.py", line 260, in calc_cond_uncond_batch
    output = model_options['model_function_wrapper'](model.apply_model, {"input": input_x, "timestep": timestep_, "c": c, "cond_or_uncond": cond_or_uncond}).chunk(batch_chunks)
  File "D:\dan4ik-forge\extensions\sd-forge-ic-light\libiclight\ic_light_nodes.py", line 61, in wrapper_func
    return existing_wrapper(unet_apply, params=apply_c_concat(params))
  File "D:\dan4ik-forge\extensions\sd-forge-ic-light\libiclight\ic_light_nodes.py", line 53, in unet_dummy_apply
    return unet_apply(x=params["input"], t=params["timestep"], **params["c"])
  File "D:\dan4ik-forge\ldm_patched\modules\model_base.py", line 90, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\dan4ik-forge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 886, in forward
    h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
  File "D:\dan4ik-forge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 61, in forward_timestep_embed
    x = layer(x)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\dan4ik-forge\ldm_patched\modules\ops.py", line 114, in forward
    return super().forward(*args, **kwargs)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\conv.py", line 460, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "D:\dan4ik-forge\venv\lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [320, 4, 3, 3], expected input[2, 8, 64, 64] to have 4 channels, but got 8 channels instead
Given groups=1, weight of size [320, 4, 3, 3], expected input[2, 8, 64, 64] to have 4 channels, but got 8 channels instead

Steps to reproduce the problem

1) Install the IC Light extension 2) Upload a test image, choose any model 3) Run generate

[Panchovix]

Hi there, thanks for the report.

If I'm not wrong, IC Light doesn't work since some time ago on dev_upstream https://github.com/Panchovix/stable-diffusion-webui-reForge/issues/30. This is because the extension uses the code form old forge, and dev_upstream has upstream changes that breaks it.

That means that an extension update is needed.

Now, you mention it worked before x commit? Are you sure you weren't on main branch?

[dan4ik94]

That means that an extension update is needed.

Now, you mention it worked before x commit? Are you sure you weren't on main branch?

yes, I'm using the dev_upstream branch, never switched to main. I found you already reported the same issue in the ic light repo, let's wait I guess, but I don't think huchenlei will fix it anytime soon.

Maybe we can do something from our side, there should be some "hacky" way to fix it.

[Newtrial99]

Sorry to chime in here, but could @Panchovix raise the same issue that was brought up in the "huchenlei" IC-Light repository on Haoming's repository at https://github.com/Haoming02/sd-forge-ic-light? Simply copying and pasting the issue would suffice. Haoming's repository is more recent and would likely be more active and responsive to the request. Additionally, your input, @Panchovix , would carry more weight and you could address any follow-up questions regarding your backend with much clarity. Thank you.

[Panchovix]

@Newtrial99 Done! Opened a new issue asking for support for dev_upstream branch with some debugging info.

[Newtrial99]

@Newtrial99 Done! Opened a new issue asking for support for dev_upstream branch with some debugging info.

it seems the IC light from Haoming now has reforge support and it works, thanks for raising the issue there, and thankyou for all the work you do here.

[Panchovix]

Hi there guys, @Haoming02 pushed a new update and now IC Light works on dev_upstream. Many thanks!