Open mm65zzz opened 1 month ago
I update the scripts to fix it, please try it again, follow the instructions.
Hi @xinsir6 and @mm65zzz,
I am still having similar / same problem in promax/controlnet_union_test_tile_deblur.py
after this change:
File "/home/XXX/1337/ControlNetPlus/promax/../models/controlnet_union.py", line 846, in forward
control_emb = self.control_add_embedding(control_embeds)
File "/home/XXX/miniconda3/envs/controlplus/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/XXX/miniconda3/envs/controlplus/lib/python3.8/site-packages/diffusers/models/embeddings.py", line 227, in forward
sample = self.linear_1(sample)
File "/home/XXX/miniconda3/envs/controlplus/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/XXX/miniconda3/envs/controlplus/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x2048 and 1536x1280)
Do you know what could be an issue?
First of all, thank you for your outstanding contribution! However, I encountered a problem when using the promax model. There is a mismatch in the expected shape of the tensor weights when trying to load a pretrained model using the
ControlNetModel_Union
class from thediffusion_pytorch_model_promax.safetensors
. Specifically, thecontrol_add_embedding.linear_1.weight
tensor has an expected shape of(1280, 1536)
but the actual shape is(1280, 2048)
.This might be because
control_add_embedding
was not initialized using the parameternum_control_type
and is still using "6".https://github.com/xinsir6/ControlNetPlus/blob/ba6c35b62e9df4c8f3b6429c4844ecc92685c8ec/models/controlnet_union.py#L409C1-L409C100
When I changed the code to
self.control_add_embedding = TimestepEmbedding(addition_time_embed_dim * num_control_type, time_embed_dim)
it loaded correctly.