size mismatch for model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 768]) from checkpoint, the shape in current model is torch.Size([320, 1280]).

rinongal / textual_inversion

MIT License

2.9k stars 279 forks source link

size mismatch for model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 768]) from checkpoint, the shape in current model is torch.Size([320, 1280]). #73

Closed mengen-li closed 2 years ago

mengen-li commented 2 years ago

how to solve

rinongal commented 2 years ago

Is this happening during inference (when running txt2img.py etc.)? It looks like you're trying to load a stable diffusion embedding into an LDM model. Please make sure you're using the correct inference script and configuration file.

mengen-li commented 2 years ago

train is: "python main.py --base configs/stable-diffusion/v1-finetune_lowmemory.yaml -t --actual_resume models/ldm/text2img-large/model.ckpt -n sanna --data_root D:\Projects\AIPaint\SDTrain\Stable-textual-inversion_win\SD\women --no-test true --gpus 1 --init-word woman"

paint is: ">python txt2img.py --ddim_eta 0.0 --n_samples 8 --n_iter 2 --scale 10.0 --ddim_steps 50 --embedding_path "D:\Projects\AIPaint\SDTrain\textual_inversion\embeddings\woman01.pt" --ckpt_path "D:\Projects\AIPaint\SDTrain\textual_inversion\models\model.ckpt" --prompt "a woman01"" model.ckpt is from stabel diffusion v1.4

rinongal commented 2 years ago

--actual_resume models/ldm/text2img-large/model.ckpt implies that you're loading the LDM model, but you're using --base configs/stable-diffusion/v1-finetune_lowmemory.yaml which is a stable-diffusion and not a latent-diffusion config.

Given your inference command, I'm guessing you want to train and run stable-diffusion, so you should change the training command to use --actual_resume "D:\Projects\AIPaint\SDTrain\textual_inversion\models\model.ckpt"