search

harlanhong / CVPR2022-DaGAN

Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation
https://harlanhong.github.io/publications/dagan.html
Other
958 stars 125 forks source link

Error occurs when I change model to SPADE_DaGAN_vox_adv_256.pth.tar #18

Closed twilight0718 closed 2 years ago

twilight0718 commented 2 years ago

As I use DaGAN_vox_adv_256.pth.tar as my pretrained model, the result is not very well. Therefore I want to change model to SPADE_DaGAN_vox_adv_256.pth.tar but error as followed occurs:

RuntimeError: Error(s) in loading state_dict for DepthAwareGenerator:
        Missing key(s) in state_dict: "up_blocks.0.conv.weight", "up_blocks.0.conv.bias", "up_blocks.0.norm.weight", "up_blocks.0.norm.bias", "up_blocks.0.norm.running_mean", "up_blocks.0.norm.running_var", "up_blocks.1.conv.weight", "up_blocks.1.conv.bias", "up_blocks.1.norm.weight", "up_blocks.1.norm.bias", "up_blocks.1.norm.running_mean", "up_blocks.1.norm.running_var", "bottleneck.r0.conv1.weight", "bottleneck.r0.conv1.bias", "bottleneck.r0.conv2.weight", "bottleneck.r0.conv2.bias", "bottleneck.r0.norm1.weight", "bottleneck.r0.norm1.bias", "bottleneck.r0.norm1.running_mean", "bottleneck.r0.norm1.running_var", "bottleneck.r0.norm2.weight", "bottleneck.r0.norm2.bias", "bottleneck.r0.norm2.running_mean", "bottleneck.r0.norm2.running_var", "bottleneck.r1.conv1.weight", "bottleneck.r1.conv1.bias", "bottleneck.r1.conv2.weight", "bottleneck.r1.conv2.bias", "bottleneck.r1.norm1.weight", "bottleneck.r1.norm1.bias", "bottleneck.r1.norm1.running_mean", "bottleneck.r1.norm1.running_var", "bottleneck.r1.norm2.weight", "bottleneck.r1.norm2.bias", "bottleneck.r1.norm2.running_mean", "bottleneck.r1.norm2.running_var", "bottleneck.r2.conv1.weight", "bottleneck.r2.conv1.bias", "bottleneck.r2.conv2.weight", "bottleneck.r2.conv2.bias", "bottleneck.r2.norm1.weight", "bottleneck.r2.norm1.bias", "bottleneck.r2.norm1.running_mean", "bottleneck.r2.norm1.running_var", "bottleneck.r2.norm2.weight", "bottleneck.r2.norm2.bias", "bottleneck.r2.norm2.running_mean", "bottleneck.r2.norm2.running_var", "bottleneck.r3.conv1.weight", "bottleneck.r3.conv1.bias", "bottleneck.r3.conv2.weight", "bottleneck.r3.conv2.bias", "bottleneck.r3.norm1.weight", "bottleneck.r3.norm1.bias", "bottleneck.r3.norm1.running_mean", "bottleneck.r3.norm1.running_var", "bottleneck.r3.norm2.weight", "bottleneck.r3.norm2.bias", "bottleneck.r3.norm2.running_mean", "bottleneck.r3.norm2.running_var", "bottleneck.r4.conv1.weight", "bottleneck.r4.conv1.bias", "bottleneck.r4.conv2.weight", "bottleneck.r4.conv2.bias", "bottleneck.r4.norm1.weight", "bottleneck.r4.norm1.bias", "bottleneck.r4.norm1.running_mean", "bottleneck.r4.norm1.running_var", "bottleneck.r4.norm2.weight", "bottleneck.r4.norm2.bias", "bottleneck.r4.norm2.running_mean", "bottleneck.r4.norm2.running_var", "bottleneck.r5.conv1.weight", "bottleneck.r5.conv1.bias", "bottleneck.r5.conv2.weight", "bottleneck.r5.conv2.bias", "bottleneck.r5.norm1.weight", "bottleneck.r5.norm1.bias", "bottleneck.r5.norm1.running_mean", "bottleneck.r5.norm1.running_var", "bottleneck.r5.norm2.weight", "bottleneck.r5.norm2.bias", "bottleneck.r5.norm2.running_mean", "bottleneck.r5.norm2.running_var", "final.weight", "final.bias". 
        Unexpected key(s) in state_dict: "decoder.compress.weight", "decoder.compress.bias", "decoder.fc.weight", "decoder.fc.bias", "decoder.G_middle_0.conv_0.bias", "decoder.G_middle_0.conv_0.weight_orig", "decoder.G_middle_0.conv_0.weight_u", "decoder.G_middle_0.conv_0.weight_v", "decoder.G_middle_0.conv_1.bias", "decoder.G_middle_0.conv_1.weight_orig", "decoder.G_middle_0.conv_1.weight_u", "decoder.G_middle_0.conv_1.weight_v", "decoder.G_middle_0.norm_0.mlp_shared.0.weight", "decoder.G_middle_0.norm_0.mlp_shared.0.bias", "decoder.G_middle_0.norm_0.mlp_gamma.weight", "decoder.G_middle_0.norm_0.mlp_gamma.bias", "decoder.G_middle_0.norm_0.mlp_beta.weight", "decoder.G_middle_0.norm_0.mlp_beta.bias", "decoder.G_middle_0.norm_1.mlp_shared.0.weight", "decoder.G_middle_0.norm_1.mlp_shared.0.bias", "decoder.G_middle_0.norm_1.mlp_gamma.weight", "decoder.G_middle_0.norm_1.mlp_gamma.bias", "decoder.G_middle_0.norm_1.mlp_beta.weight", "decoder.G_middle_0.norm_1.mlp_beta.bias", "decoder.G_middle_1.conv_0.bias", "decoder.G_middle_1.conv_0.weight_orig", "decoder.G_middle_1.conv_0.weight_u", "decoder.G_middle_1.conv_0.weight_v", "decoder.G_middle_1.conv_1.bias", "decoder.G_middle_1.conv_1.weight_orig", "decoder.G_middle_1.conv_1.weight_u", "decoder.G_middle_1.conv_1.weight_v", "decoder.G_middle_1.norm_0.mlp_shared.0.weight", "decoder.G_middle_1.norm_0.mlp_shared.0.bias", "decoder.G_middle_1.norm_0.mlp_gamma.weight", "decoder.G_middle_1.norm_0.mlp_gamma.bias", "decoder.G_middle_1.norm_0.mlp_beta.weight", "decoder.G_middle_1.norm_0.mlp_beta.bias", "decoder.G_middle_1.norm_1.mlp_shared.0.weight", "decoder.G_middle_1.norm_1.mlp_shared.0.bias", "decoder.G_middle_1.norm_1.mlp_gamma.weight", "decoder.G_middle_1.norm_1.mlp_gamma.bias", "decoder.G_middle_1.norm_1.mlp_beta.weight", "decoder.G_middle_1.norm_1.mlp_beta.bias", "decoder.G_middle_2.conv_0.bias", "decoder.G_middle_2.conv_0.weight_orig", "decoder.G_middle_2.conv_0.weight_u", "decoder.G_middle_2.conv_0.weight_v", "decoder.G_middle_2.conv_1.bias", "decoder.G_middle_2.conv_1.weight_orig", "decoder.G_middle_2.conv_1.weight_u", "decoder.G_middle_2.conv_1.weight_v", "decoder.G_middle_2.norm_0.mlp_shared.0.weight", "decoder.G_middle_2.norm_0.mlp_shared.0.bias", "decoder.G_middle_2.norm_0.mlp_gamma.weight", "decoder.G_middle_2.norm_0.mlp_gamma.bias", "decoder.G_middle_2.norm_0.mlp_beta.weight", "decoder.G_middle_2.norm_0.mlp_beta.bias", "decoder.G_middle_2.norm_1.mlp_shared.0.weight", "decoder.G_middle_2.norm_1.mlp_shared.0.bias", "decoder.G_middle_2.norm_1.mlp_gamma.weight", "decoder.G_middle_2.norm_1.mlp_gamma.bias", "decoder.G_middle_2.norm_1.mlp_beta.weight", "decoder.G_middle_2.norm_1.mlp_beta.bias", "decoder.up_0.conv_0.bias", "decoder.up_0.conv_0.weight_orig", "decoder.up_0.conv_0.weight_u", "decoder.up_0.conv_0.weight_v", "decoder.up_0.conv_1.bias", "decoder.up_0.conv_1.weight_orig", "decoder.up_0.conv_1.weight_u", "decoder.up_0.conv_1.weight_v", "decoder.up_0.conv_s.weight_orig", "decoder.up_0.conv_s.weight_u", "decoder.up_0.conv_s.weight_v", "decoder.up_0.norm_0.mlp_shared.0.weight", "decoder.up_0.norm_0.mlp_shared.0.bias", "decoder.up_0.norm_0.mlp_gamma.weight", "decoder.up_0.norm_0.mlp_gamma.bias", "decoder.up_0.norm_0.mlp_beta.weight", "decoder.up_0.norm_0.mlp_beta.bias", "decoder.up_0.norm_1.mlp_shared.0.weight", "decoder.up_0.norm_1.mlp_shared.0.bias", "decoder.up_0.norm_1.mlp_gamma.weight", "decoder.up_0.norm_1.mlp_gamma.bias", "decoder.up_0.norm_1.mlp_beta.weight", "decoder.up_0.norm_1.mlp_beta.bias", "decoder.up_0.norm_s.mlp_shared.0.weight", "decoder.up_0.norm_s.mlp_shared.0.bias", "decoder.up_0.norm_s.mlp_gamma.weight", "decoder.up_0.norm_s.mlp_gamma.bias", "decoder.up_0.norm_s.mlp_beta.weight", "decoder.up_0.norm_s.mlp_beta.bias", "decoder.up_1.conv_0.bias", "decoder.up_1.conv_0.weight_orig", "decoder.up_1.conv_0.weight_u", "decoder.up_1.conv_0.weight_v", "decoder.up_1.conv_1.bias", "decoder.up_1.conv_1.weight_orig", "decoder.up_1.conv_1.weight_u", "decoder.up_1.conv_1.weight_v", "decoder.up_1.conv_s.weight_orig", "decoder.up_1.conv_s.weight_u", "decoder.up_1.conv_s.weight_v", "decoder.up_1.norm_0.mlp_shared.0.weight", "decoder.up_1.norm_0.mlp_shared.0.bias", "decoder.up_1.norm_0.mlp_gamma.weight", "decoder.up_1.norm_0.mlp_gamma.bias", "decoder.up_1.norm_0.mlp_beta.weight", "decoder.up_1.norm_0.mlp_beta.bias", "decoder.up_1.norm_1.mlp_shared.0.weight", "decoder.up_1.norm_1.mlp_shared.0.bias", "decoder.up_1.norm_1.mlp_gamma.weight", "decoder.up_1.norm_1.mlp_gamma.bias", "decoder.up_1.norm_1.mlp_beta.weight", "decoder.up_1.norm_1.mlp_beta.bias", "decoder.up_1.norm_s.mlp_shared.0.weight", "decoder.up_1.norm_s.mlp_shared.0.bias", "decoder.up_1.norm_s.mlp_gamma.weight", "decoder.up_1.norm_s.mlp_gamma.bias", "decoder.up_1.norm_s.mlp_beta.weight", "decoder.up_1.norm_s.mlp_beta.bias", "decoder.conv_img.weight", "decoder.conv_img.bias".

It seems that those two pretrained model have different structures, should I change something in demo.py or vox-adv-256.yaml? Looking forward to your reply, Thx a lot!

twilight0718 commented 2 years ago

Well, I should double check the closed issue in advance, sorry for disrupption.