budui
commented
7 months ago Can you share the code you used? It seems that the reason is that you directly gave the T5 features instead of the ella-converted features to Animatediff.
Open anthonyyuan opened 7 months ago
budui
commented
7 months ago Can you share the code you used? It seems that the reason is that you directly gave the T5 features instead of the ella-converted features to Animatediff.
key = attn.to_k(encoder_hidden_states) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/peft/tuners/lora/layer.py", line 495, in forward result = self.base_layer(x, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 116, in forward return F.linear(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: mat1 and mat2 shapes cannot be multiplied (2480x2048 and 768x320)