search

huchenlei / sd-forge-ic-light

A1111/SD Forge extension for IC-Light
Apache License 2.0
190 stars 5 forks source link

Support for reForge webui dev_upstream branch #36

Open Panchovix opened 2 months ago

Panchovix commented 2 months ago

Hi there, thanks for your all hard work!

I did fork and do updates up to A1111/Comfy upstream here https://github.com/Panchovix/stable-diffusion-webui-reForge/tree/dev_upstream

Since I changed the code to upstream comfy for ldm_patched\ldm\modules\diffusionmodules\openaimodel.py and conv2d on ldm_patched/modules/ops.py, now there is some issues.

Error is

Downloading data from 'https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net_human_seg.onnx' to file 'G:\Stable difussion\stable-diffusion-webui\models\u2net\u2net_human_seg.onnx'.
100%|###############################################| 176M/176M [00:00<?, ?B/s]
To load target model AutoencoderKL
Begin to load 1 model
Moving model(s) has taken 0.03 seconds
To load target model AutoencoderKL
Begin to load 1 model
Reuse 1 loaded models
Moving model(s) has taken 0.00 seconds
To load target model BaseModel
Begin to load 1 model
WARNING:root:WARNING SHAPE MISMATCH diffusion_model.input_blocks.0.0.weight WEIGHT NOT MERGED torch.Size([320, 12, 3, 3]) != torch.Size([320, 4, 3, 3])
Moving model(s) has taken 0.13 seconds
  0%|                                                                                           | 0/25 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules_forge\main_thread.py", line 37, in loop
    task.work()
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules_forge\main_thread.py", line 26, in work
    self.result = self.func(*self.args, **self.kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\txt2img.py", line 110, in txt2img_function
    processed = processing.process_images(p)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\processing.py", line 815, in process_images
    res = process_images_inner(p)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\processing.py", line 988, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\processing.py", line 1362, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\sd_samplers_kdiffusion.py", line 236, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\sd_samplers_common.py", line 274, in launch_sampling
    return func()
           ^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\sd_samplers_kdiffusion.py", line 236, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
    denoised = model(x, sigmas[i] * s_in, **extra_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\module.py", line 1714, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\module.py", line 1725, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\modules\sd_samplers_cfg_denoiser.py", line 369, in forward
    denoised = sampling_function(model, x, sigma, uncond_patched, cond_patched, cond_scale, model_options, seed)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\ldm_patched\modules\samplers.py", line 290, in sampling_function
    cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\ldm_patched\modules\samplers.py", line 257, in calc_cond_uncond_batch
    output = model_options['model_function_wrapper'](model.apply_model, {"input": input_x, "timestep": timestep_, "c": c, "cond_or_uncond": cond_or_uncond}).chunk(batch_chunks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\extensions\sd-forge-ic-light\libiclight\ic_light_nodes.py", line 61, in wrapper_func
    return existing_wrapper(unet_apply, params=apply_c_concat(params))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\extensions\sd-forge-ic-light\libiclight\ic_light_nodes.py", line 53, in unet_dummy_apply
    return unet_apply(x=params["input"], t=params["timestep"], **params["c"])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\ldm_patched\modules\model_base.py", line 118, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\module.py", line 1714, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\module.py", line 1725, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 859, in forward
    h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 57, in forward_timestep_embed
    x = layer(x)
        ^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\module.py", line 1714, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\module.py", line 1725, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui-reForge\ldm_patched\modules\ops.py", line 137, in forward
    return super().forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\conv.py", line 549, in forward
    return self._conv_forward(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\Stable difussion\stable-diffusion-webui\venv\Lib\site-packages\torch\nn\modules\conv.py", line 544, in _conv_forward
    return F.conv2d(
           ^^^^^^^^^
RuntimeError: Given groups=1, weight of size [320, 4, 3, 3], expected input[2, 12, 136, 112] to have 4 channels, but got 12 channels instead
Given groups=1, weight of size [320, 4, 3, 3], expected input[2, 12, 136, 112] to have 4 channels, but got 12 channels instead

For VAE it does get it from def clone(self) on https://github.com/Panchovix/stable-diffusion-webui-reForge/blob/99cb084fbdac6e0084052b82bbe31140867b9213/ldm_patched/modules/sd.py#L293