LiJT
commented
6 months ago I'm sorry, it is for SDXL model only, sorry for wasting your time
Closed LiJT closed 6 months ago
LiJT
commented
6 months ago I'm sorry, it is for SDXL model only, sorry for wasting your time
otichy
commented
4 months ago I'm sorry, it is for SDXL model only, sorry for wasting your time
Hi, would you let me know of a config that worked for you?
I am using sdxl base model (but also tried with SD1.5) and I could not make it work with any combination of the ControlNet preprocessor and model - I always got the size mismatch (different sizes with different models, but I am always generating 1024x1024 and my control image is the same size). Thank you!
jacquesfeng123
commented
4 months ago 2024-05-07 17:33:54,817 - ControlNet - INFO - ControlNet Input Mode: InputMode.SIMPLE 2024-05-07 17:33:54,853 - ControlNet - INFO - Using preprocessor: CLIP-ViT-H (IPAdapter) 2024-05-07 17:33:54,854 - ControlNet - INFO - preprocessor resolution = 832 2024-05-07 17:33:54,861 - ControlNet - INFO - Current ControlNet IPAdapterPatcher: /home/lam/stable-diffusion-webui/models/ControlNet/ip-adapter-plus_sdxl_vit-h.safetensors INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting. Error running process_before_every_sampling: /home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/scripts/controlnet.py Traceback (most recent call last): File "/home/lam/stable-diffusion-webui-forge/modules/scripts.py", line 835, in process_before_every_sampling script.process_before_every_sampling(p, script_args, kwargs) File "/home/lam/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/scripts/controlnet.py", line 555, in process_before_every_sampling self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, *kwargs) File "/home/lam/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/scripts/controlnet.py", line 501, in process_unit_before_every_sampling params.model.process_before_every_sampling(p, cond, mask, *args, **kwargs) File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_ipadapter/scripts/forge_ipadapter.py", line 147, in process_before_every_sampling unet = opIPAdapterApply( File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_ipadapter/lib_ipadapter/IPAdapterPlus.py", line 723, in apply_ipadapter self.ipadapter = IPAdapter( File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_ipadapter/lib_ipadapter/IPAdapterPlus.py", line 286, in init self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "/home/lam/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1664]).
hi, we got carried away. I am facing this issue too.
I should be using the correct preprocessor and model.
model redownloaed from the link the author gave, and preprocessor CLIP-ViT-H (IPAdapter), I am using SDXL endpoint
I am having the same issue too :(
Checklist
What happened?
I have received this error message when i tried to use the CLIP-ViT-bigG (IPAdapter) Preprocessor in Forge
Error running process_before_every_sampling: E:\SDWebUI_Forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py Traceback (most recent call last): File "E:\SDWebUI_Forge\webui\modules\scripts.py", line 835, in process_before_every_sampling script.process_before_every_sampling(p, script_args, kwargs) File "E:\SDWebUI_Forge\system\python\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "E:\SDWebUI_Forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 555, in process_before_every_sampling self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, *kwargs) File "E:\SDWebUI_Forge\system\python\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "E:\SDWebUI_Forge\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 501, in process_unit_before_every_sampling params.model.process_before_every_sampling(p, cond, mask, *args, **kwargs) File "E:\SDWebUI_Forge\webui\extensions-builtin\sd_forge_ipadapter\scripts\forge_ipadapter.py", line 147, in process_before_every_sampling unet = opIPAdapterApply( File "E:\SDWebUI_Forge\webui\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 723, in apply_ipadapter self.ipadapter = IPAdapter( File "E:\SDWebUI_Forge\webui\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 286, in init self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "E:\SDWebUI_Forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1664]).
I have no idea whats going, other preprocessor works just fine, but not with CLIP-ViT-bigG
Steps to reproduce the problem
What should have happened?
There should not be any errors....
What browsers do you use to access the UI ?
Google Chrome
Sysinfo
s
Console logs
Additional information
No response