CyberAgentAILab / TANGO

Official implementation of the paper "TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio-Motion Embedding and Diffusion Interpolation"
https://pantomatrix.github.io/TANGO/
Other
358 stars 77 forks source link

How can we use high-quality modes? #1

Open mnbv7758 opened 1 month ago

mnbv7758 commented 1 month ago

How can we use high-quality modes?

H-Liu1997 commented 1 month ago

@mnbv7758 Thanks for your interest!

if you want to try high-quality mode, please consider:

  1. use the saved HD video in line 369 in app.py
  2. replace line 370 - line 374 with a HD lipsync model, in our HD demo we use in-lab model. for open-source version, I suggest you take a look on wav2lip-hd
mnbv7758 commented 3 weeks ago

Can using wav2lip-hd solve the "frame skipping" problem?

H-Liu1997 commented 3 weeks ago

@mnbv7758 frame skipping is caused by threshold to create the graph in

https://github.com/CyberAgentAILab/TANGO/blob/e24a2f4c64f4e9addb57aad1ebb8cdaf2eeada6c/create_graph.py#L232-L235

you can make the threshold be higher

mnbv7758 commented 2 weeks ago

I replaced the model with wav2lip384 and it reported the following error:

Moviepy - Done ! Moviepy - video ready ./outputs/gradio/test_0/retrieved_motions_0/audio_0_retri_0.mp4 Using cuda for inference. Load checkpoint from: ./Wav2Lip/checkpoints/checkpoint_step000318000.pth Traceback (most recent call last): File "/AI/TANGO/./Wav2Lip/inference.py", line 322, in do_load(args.checkpoint_path) File "/AI/TANGO/./Wav2Lip/inference.py", line 294, in do_load model = load_model(checkpoint_path) File "/AI/TANGO/./Wav2Lip/inference.py", line 167, in load_model model.load_state_dict(new_s) File "/AI/miniconda3/envs/TANGO/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Wav2Lip: Missing key(s) in state_dict: "face_encoder_blocks.2.3.conv_block.0.weight", "face_encoder_blocks.2.3.conv_block.0.bias", "face_encoder_blocks.2.3.conv_block.1.weight", "face_encoder_blocks.2.3.conv_block.1.bias", "face_encoder_blocks.2.3.conv_block.1.running_mean", "face_encoder_blocks.2.3.conv_block.1.running_var". Unexpected key(s) in state_dict: "audio_refine.0.conv_block.0.weight", "audio_refine.0.conv_block.0.bias", "audio_refine.0.conv_block.1.weight", "audio_refine.0.conv_bl ock.1.bias", "audio_refine.0.conv_block.1.running_mean", "audio_refine.0.conv_block.1.running_var", "audio_refine.0.conv_block.1.num_batches_tracked", "audio_refine.1.conv_bloc k.0.weight", "audio_refine.1.conv_block.0.bias", "audio_refine.1.conv_block.1.weight", "audio_refine.1.conv_block.1.bias", "audio_refine.1.conv_block.1.running_mean", "audio_re fine.1.conv_block.1.running_var", "audio_refine.1.conv_block.1.num_batches_tracked", "face_encoder_blocks.7.0.conv_block.0.weight", "face_encoder_blocks.7.0.conv_block.0.bias", "face_encoder_blocks.7.0.conv_block.1.weight", "face_encoder_blocks.7.0.conv_block.1.bias", "face_encoder_blocks.7.0.conv_block.1.running_mean", "face_encoderblocks.7.0.conv block.1.running_var", "face_encoder_blocks.7.0.conv_block.1.num_batches_tracked", "face_encoder_blocks.7.1.conv_block.0.weight", "face_encoder_blocks.7.1.conv_block.0.bias", "f ace_encoder_blocks.7.1.conv_block.1.weight", "face_encoder_blocks.7.1.conv_block.1.bias", "face_encoder_blocks.7.1.conv_block.1.running_mean", "face_encoder_blocks.7.1.conv_blo ck.1.running_var", "face_encoder_blocks.7.1.conv_block.1.num_batches_tracked", "face_encoder_blocks.7.2.conv_block.0.weight", "face_encoder_blocks.7.2.conv_block.0.bias", "face _encoder_blocks.7.2.conv_block.1.weight", "face_encoder_blocks.7.2.conv_block.1.bias", "face_encoder_blocks.7.2.conv_block.1.running_mean", "face_encoder_blocks.7.2.conv_block. 1.running_var", "face_encoder_blocks.7.2.conv_block.1.num_batches_tracked", "face_encoder_blocks.8.0.conv_block.0.weight", "face_encoder_blocks.8.0.conv_block.0.bias", "face_en coder_blocks.8.0.conv_block.1.weight", "face_encoder_blocks.8.0.conv_block.1.bias", "face_encoder_blocks.8.0.conv_block.1.running_mean", "face_encoder_blocks.8.0.conv_block.1.r unning_var", "face_encoder_blocks.8.0.conv_block.1.num_batches_tracked", "face_encoder_blocks.8.1.conv_block.0.weight", "face_encoder_blocks.8.1.conv_block.0.bias", "face_encod er_blocks.8.1.conv_block.1.weight", "face_encoder_blocks.8.1.conv_block.1.bias", "face_encoder_blocks.8.1.conv_block.1.running_mean", "face_encoder_blocks.8.1.conv_block.1.runn ing_var", "face_encoder_blocks.8.1.conv_block.1.num_batches_tracked", "face_encoder_blocks.3.3.conv_block.0.weight", "face_encoder_blocks.3.3.conv_block.0.bias", "faceencoder blocks.3.3.conv_block.1.weight", "face_encoder_blocks.3.3.conv_block.1.bias", "face_encoder_blocks.3.3.conv_block.1.running_mean", "face_encoder_blocks.3.3.conv_block.1.running _var", "face_encoder_blocks.3.3.conv_block.1.num_batches_tracked", "face_encoder_blocks.5.2.conv_block.0.weight", "face_encoder_blocks.5.2.conv_block.0.bias", "face_encoder_blo cks.5.2.conv_block.1.weight", "face_encoder_blocks.5.2.conv_block.1.bias", "face_encoder_blocks.5.2.conv_block.1.running_mean", "face_encoder_blocks.5.2.conv_block.1.running_va r", "face_encoder_blocks.5.2.conv_block.1.num_batches_tracked", "face_encoder_blocks.6.2.conv_block.0.weight", "face_encoder_blocks.6.2.conv_block.0.bias", "face_encoder_blocks .6.2.conv_block.1.weight", "face_encoder_blocks.6.2.conv_block.1.bias", "face_encoder_blocks.6.2.conv_block.1.running_mean", "face_encoder_blocks.6.2.conv_block.1.running_var", "face_encoder_blocks.6.2.conv_block.1.num_batches_tracked", "audio_encoder.13.conv_block.0.weight", "audio_encoder.13.conv_block.0.bias", "audio_encoder.13.conv_block.1.weight ", "audio_encoder.13.conv_block.1.bias", "audio_encoder.13.conv_block.1.running_mean", "audio_encoder.13.conv_block.1.running_var", "audio_encoder.13.conv_block.1.num_batches_t racked", "audio_encoder.14.conv_block.0.weight", "audio_encoder.14.conv_block.0.bias", "audio_encoder.14.conv_block.1.weight", "audio_encoder.14.conv_block.1.bias", "audio_enco der.14.conv_block.1.running_mean", "audio_encoder.14.conv_block.1.running_var", "audio_encoder.14.conv_block.1.num_batches_tracked", "audio_encoder.15.conv_block.0.weight", "au dio_encoder.15.conv_block.0.bias", "audio_encoder.15.conv_block.1.weight", "audio_encoder.15.conv_block.1.bias", "audio_encoder.15.conv_block.1.running_mean", "audio_encoder.15 .conv_block.1.running_var", "audio_encoder.15.conv_block.1.num_batches_tracked", "audio_encoder.16.conv_block.0.weight", "audio_encoder.16.conv_block.0.bias", "audio_encoder.16 .conv_block.1.weight", "audio_encoder.16.conv_block.1.bias", "audio_encoder.16.conv_block.1.running_mean", "audio_encoder.16.conv_block.1.running_var", "audio_encoder.16.conv_b lock.1.num_batches_tracked", "face_decoder_blocks.7.0.conv_block.0.weight", "face_decoder_blocks.7.0.conv_block.0.bias", "face_decoder_blocks.7.0.conv_block.1.weight", "face_de coder_blocks.7.0.conv_block.1.bias", "face_decoder_blocks.7.0.conv_block.1.running_mean", "face_decoder_blocks.7.0.conv_block.1.running_var", "face_decoder_blocks.7.0.conv_bloc k.1.num_batches_tracked", "face_decoder_blocks.7.1.conv_block.0.weight", "face_decoder_blocks.7.1.conv_block.0.bias", "face_decoder_blocks.7.1.conv_block.1.weight", "face_decod er_blocks.7.1.conv_block.1.bias", "face_decoder_blocks.7.1.conv_block.1.running_mean", "face_decoder_blocks.7.1.conv_block.1.running_var", "face_decoder_blocks.7.1.conv_block.1 .num_batches_tracked", "face_decoder_blocks.7.2.conv_block.0.weight", "face_decoder_blocks.7.2.conv_block.0.bias", "face_decoder_blocks.7.2.conv_block.1.weight", "facedecoder blocks.7.2.conv_block.1.bias", "face_decoder_blocks.7.2.conv_block.1.running_mean", "face_decoder_blocks.7.2.conv_block.1.running_var", "face_decoder_blocks.7.2.conv_block.1.nu m_batches_tracked", "face_decoder_blocks.8.0.conv_block.0.weight", "face_decoder_blocks.8.0.conv_block.0.bias", "face_decoder_blocks.8.0.conv_block.1.weight", "face_decoder_blo cks.8.0.conv_block.1.bias", "face_decoder_blocks.8.0.conv_block.1.running_mean", "face_decoder_blocks.8.0.conv_block.1.running_var", "face_decoder_blocks.8.0.conv_block.1.num_b atches_tracked", "face_decoder_blocks.8.1.conv_block.0.weight", "face_decoder_blocks.8.1.conv_block.0.bias", "face_decoder_blocks.8.1.conv_block.1.weight", "face_decoder_blocks .8.1.conv_block.1.bias", "face_decoder_blocks.8.1.conv_block.1.running_mean", "face_decoder_blocks.8.1.conv_block.1.running_var", "face_decoder_blocks.8.1.conv_block.1.num_batc hes_tracked", "face_decoder_blocks.8.2.conv_block.0.weight", "face_decoder_blocks.8.2.conv_block.0.bias", "face_decoder_blocks.8.2.conv_block.1.weight", "face_decoder_blocks.8. 2.conv_block.1.bias", "face_decoder_blocks.8.2.conv_block.1.running_mean", "face_decoder_blocks.8.2.conv_block.1.running_var", "face_decoder_blocks.8.2.conv_block.1.num_batches _tracked". size mismatch for face_encoder_blocks.0.0.conv_block.0.weight: copying a param with shape torch.Size([8, 6, 7, 7]) from checkpoint, the shape in current model is torch. Size([16, 6, 7, 7]). size mismatch for face_encoder_blocks.0.0.conv_block.0.bias: copying a param with shape torch.Size([8]) from checkpoint, the shape in current model is torch.Size([16]). size mismatch for face_encoder_blocks.0.0.conv_block.1.weight: copying a param with shape torch.Size([8]) from checkpoint, the shape in current model is torch.Size([16] ). size mismatch for face_encoder_blocks.0.0.conv_block.1.bias: copying a param with shape torch.Size([8]) from checkpoint, the shape in current model is torch.Size([16]). size mismatch for face_encoder_blocks.0.0.conv_block.1.running_mean: copying a param with shape torch.Size([8]) from checkpoint, the shape in current model is torch.Siz e([16]). size mismatch for face_encoder_blocks.0.0.conv_block.1.running_var: copying a param with shape torch.Size([8]) from checkpoint, the shape in current model is torch.Size ([16]). size mismatch for face_encoder_blocks.1.0.conv_block.0.weight: copying a param with shape torch.Size([16, 8, 3, 3]) from checkpoint, the shape in current model is torch .Size([32, 16, 3, 3]). size mismatch for face_encoder_blocks.1.0.conv_block.0.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]) . size mismatch for face_encoder_blocks.1.0.conv_block.1.weight: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32 ]). size mismatch for face_encoder_blocks.1.0.conv_block.1.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]) . size mismatch for face_encoder_blocks.1.0.conv_block.1.running_mean: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Si ze([32]). size mismatch for face_encoder_blocks.1.0.conv_block.1.running_var: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Siz e([32]). size mismatch for face_encoder_blocks.1.1.conv_block.0.weight: copying a param with shape torch.Size([16, 16, 3, 3]) from checkpoint, the shape in current model is torc h.Size([32, 32, 3, 3]). size mismatch for face_encoder_blocks.1.1.conv_block.0.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]) . size mismatch for face_encoder_blocks.1.1.conv_block.1.weight: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32 ]). size mismatch for face_encoder_blocks.1.1.conv_block.1.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]) . size mismatch for face_encoder_blocks.1.1.conv_block.1.running_mean: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Si ze([32]). size mismatch for face_encoder_blocks.1.1.conv_block.1.running_var: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Siz e([32]). size mismatch for face_encoder_blocks.1.2.conv_block.0.weight: copying a param with shape torch.Size([16, 16, 3, 3]) from checkpoint, the shape in current model is torc h.Size([32, 32, 3, 3]). size mismatch for face_encoder_blocks.1.2.conv_block.0.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]) . size mismatch for face_encoder_blocks.1.2.conv_block.1.weight: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32 ]). size mismatch for face_encoder_blocks.1.2.conv_block.1.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]) . size mismatch for face_encoder_blocks.1.2.conv_block.1.running_mean: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Si ze([32]). size mismatch for face_encoder_blocks.1.2.conv_block.1.running_var: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Siz e([32]). size mismatch for face_encoder_blocks.2.0.conv_block.0.weight: copying a param with shape torch.Size([32, 16, 3, 3]) from checkpoint, the shape in current model is torc h.Size([64, 32, 3, 3]). size mismatch for face_encoder_blocks.2.0.conv_block.0.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]) . size mismatch for face_encoder_blocks.2.0.conv_block.1.weight: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64 ]). size mismatch for face_encoder_blocks.2.0.conv_block.1.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]) . size mismatch for face_encoder_blocks.2.0.conv_block.1.running_mean: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Si ze([64]). size mismatch for face_encoder_blocks.2.0.conv_block.1.running_var: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Siz e([64]). size mismatch for face_encoder_blocks.2.1.conv_block.0.weight: copying a param with shape torch.Size([32, 32, 3, 3]) from checkpoint, the shape in current model is torc h.Size([64, 64, 3, 3]). size mismatch for face_encoder_blocks.2.1.conv_block.0.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]) . size mismatch for face_encoder_blocks.2.1.conv_block.1.weight: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64 ]). size mismatch for face_encoder_blocks.2.1.conv_block.1.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]) . size mismatch for face_encoder_blocks.2.1.conv_block.1.running_mean: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Si ze([64]). size mismatch for face_encoder_blocks.2.1.conv_block.1.running_var: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Siz e([64]). size mismatch for face_encoder_blocks.2.2.conv_block.0.weight: copying a param with shape torch.Size([32, 32, 3, 3]) from checkpoint, the shape in current model is torc h.Size([64, 64, 3, 3]). size mismatch for face_encoder_blocks.2.2.conv_block.0.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]) . size mismatch for face_encoder_blocks.2.2.conv_block.1.weight: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64 ]). size mismatch for face_encoder_blocks.2.2.conv_block.1.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]) . size mismatch for face_encoder_blocks.2.2.conv_block.1.running_mean: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Si ze([64]). size mismatch for face_encoder_blocks.2.2.conv_block.1.running_var: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Siz e([64]). size mismatch for face_encoder_blocks.3.0.conv_block.0.weight: copying a param with shape torch.Size([64, 32, 3, 3]) from checkpoint, the shape in current model is torc h.Size([128, 64, 3, 3]). size mismatch for face_encoder_blocks.3.0.conv_block.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128] ). size mismatch for face_encoder_blocks.3.0.conv_block.1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([12 8]). size mismatch for face_encoder_blocks.3.0.conv_block.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128] ). size mismatch for face_encoder_blocks.3.0.conv_block.1.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Si ze([128]). size mismatch for face_encoder_blocks.3.0.conv_block.1.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Siz e([128]). size mismatch for face_encoder_blocks.3.1.conv_block.0.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torc h.Size([128, 128, 3, 3]). size mismatch for face_encoder_blocks.3.1.conv_block.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128] ). size mismatch for face_encoder_blocks.3.1.conv_block.1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([12 8]). size mismatch for face_encoder_blocks.3.1.conv_block.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128] ). size mismatch for face_encoder_blocks.3.1.conv_block.1.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Si ze([128]). size mismatch for face_encoder_blocks.3.1.conv_block.1.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Siz e([128]). size mismatch for face_encoder_blocks.3.2.conv_block.0.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torc h.Size([128, 128, 3, 3]). size mismatch for face_encoder_blocks.3.2.conv_block.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128] ). size mismatch for face_encoder_blocks.3.2.conv_block.1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([12 8]). size mismatch for face_encoder_blocks.3.2.conv_block.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128] ). size mismatch for face_encoder_blocks.3.2.conv_block.1.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Si ze([128]). size mismatch for face_encoder_blocks.3.2.conv_block.1.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Siz e([128]). size mismatch for face_encoder_blocks.4.0.conv_block.0.weight: copying a param with shape torch.Size([128, 64, 3, 3]) from checkpoint, the shape in current model is tor ch.Size([256, 128, 3, 3]). size mismatch for face_encoder_blocks.4.0.conv_block.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_encoder_blocks.4.0.conv_block.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([2 56]). size mismatch for face_encoder_blocks.4.0.conv_block.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_encoder_blocks.4.0.conv_block.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.S ize([256]). size mismatch for face_encoder_blocks.4.0.conv_block.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Si ze([256]). size mismatch for face_encoder_blocks.4.1.conv_block.0.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is to rch.Size([256, 256, 3, 3]). size mismatch for face_encoder_blocks.4.1.conv_block.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_encoder_blocks.4.1.conv_block.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([2 56]). size mismatch for face_encoder_blocks.4.1.conv_block.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_encoder_blocks.4.1.conv_block.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.S ize([256]). size mismatch for face_encoder_blocks.4.1.conv_block.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Si ze([256]). size mismatch for face_encoder_blocks.4.2.conv_block.0.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is to rch.Size([256, 256, 3, 3]). size mismatch for face_encoder_blocks.4.2.conv_block.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_encoder_blocks.4.2.conv_block.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([2 56]). size mismatch for face_encoder_blocks.4.2.conv_block.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_encoder_blocks.4.2.conv_block.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.S ize([256]). size mismatch for face_encoder_blocks.4.2.conv_block.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Si ze([256]). size mismatch for face_encoder_blocks.5.0.conv_block.0.weight: copying a param with shape torch.Size([256, 128, 3, 3]) from checkpoint, the shape in current model is to rch.Size([512, 256, 3, 3]). size mismatch for face_encoder_blocks.5.0.conv_block.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512 ]). size mismatch for face_encoder_blocks.5.0.conv_block.1.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([5 12]). size mismatch for face_encoder_blocks.5.0.conv_block.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512 ]). size mismatch for face_encoder_blocks.5.0.conv_block.1.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_encoder_blocks.5.0.conv_block.1.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Si ze([512]). size mismatch for face_encoder_blocks.5.1.conv_block.0.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is to rch.Size([512, 512, 3, 3]). size mismatch for face_encoder_blocks.5.1.conv_block.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512 ]). size mismatch for face_encoder_blocks.5.1.conv_block.1.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([5 12]). size mismatch for face_encoder_blocks.5.1.conv_block.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512 ]). size mismatch for face_encoder_blocks.5.1.conv_block.1.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_encoder_blocks.5.1.conv_block.1.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Si ze([512]). size mismatch for face_encoder_blocks.6.0.conv_block.0.weight: copying a param with shape torch.Size([512, 256, 3, 3]) from checkpoint, the shape in current model is to rch.Size([512, 512, 3, 3]). size mismatch for face_encoder_blocks.6.1.conv_block.0.weight: copying a param with shape torch.Size([512, 512, 3, 3]) from checkpoint, the shape in current model is to rch.Size([512, 512, 1, 1]). size mismatch for audio_encoder.11.conv_block.0.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is torch.Siz e([512, 256, 3, 3]). size mismatch for audio_encoder.11.conv_block.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]). size mismatch for audio_encoder.11.conv_block.1.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]). size mismatch for audio_encoder.11.conv_block.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]). size mismatch for audio_encoder.11.conv_block.1.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for audio_encoder.11.conv_block.1.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512 ]). size mismatch for audio_encoder.12.conv_block.0.weight: copying a param with shape torch.Size([512, 256, 3, 3]) from checkpoint, the shape in current model is torch.Siz e([512, 512, 1, 1]). size mismatch for face_decoder_blocks.0.0.conv_block.0.weight: copying a param with shape torch.Size([1024, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]). size mismatch for face_decoder_blocks.0.0.conv_block.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.0.0.conv_block.1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([ 512]). size mismatch for face_decoder_blocks.0.0.conv_block.1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.0.0.conv_block.1.running_mean: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch. Size([512]). size mismatch for face_decoder_blocks.0.0.conv_block.1.running_var: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_decoder_blocks.1.0.conv_block.0.weight: copying a param with shape torch.Size([2048, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([1024, 512, 3, 3]). size mismatch for face_decoder_blocks.1.0.conv_block.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.1.0.conv_block.1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([ 512]). size mismatch for face_decoder_blocks.1.0.conv_block.1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.1.0.conv_block.1.running_mean: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch. Size([512]). size mismatch for face_decoder_blocks.1.0.conv_block.1.running_var: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_decoder_blocks.1.1.conv_block.0.weight: copying a param with shape torch.Size([1024, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]). size mismatch for face_decoder_blocks.1.1.conv_block.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.1.1.conv_block.1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([ 512]). size mismatch for face_decoder_blocks.1.1.conv_block.1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.1.1.conv_block.1.running_mean: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch. Size([512]). size mismatch for face_decoder_blocks.1.1.conv_block.1.running_var: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_decoder_blocks.2.0.conv_block.0.weight: copying a param with shape torch.Size([2048, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([1024, 512, 3, 3]). size mismatch for face_decoder_blocks.2.0.conv_block.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.2.0.conv_block.1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([ 512]). size mismatch for face_decoder_blocks.2.0.conv_block.1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.2.0.conv_block.1.running_mean: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch. Size([512]). size mismatch for face_decoder_blocks.2.0.conv_block.1.running_var: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_decoder_blocks.2.1.conv_block.0.weight: copying a param with shape torch.Size([1024, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]). size mismatch for face_decoder_blocks.2.1.conv_block.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.2.1.conv_block.1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([ 512]). size mismatch for face_decoder_blocks.2.1.conv_block.1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.2.1.conv_block.1.running_mean: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch. Size([512]). size mismatch for face_decoder_blocks.2.1.conv_block.1.running_var: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_decoder_blocks.2.2.conv_block.0.weight: copying a param with shape torch.Size([1024, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]). size mismatch for face_decoder_blocks.2.2.conv_block.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.2.2.conv_block.1.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([ 512]). size mismatch for face_decoder_blocks.2.2.conv_block.1.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([51 2]). size mismatch for face_decoder_blocks.2.2.conv_block.1.running_mean: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch. Size([512]). size mismatch for face_decoder_blocks.2.2.conv_block.1.running_var: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.S ize([512]). size mismatch for face_decoder_blocks.3.0.conv_block.0.weight: copying a param with shape torch.Size([1536, 768, 3, 3]) from checkpoint, the shape in current model is t orch.Size([768, 384, 3, 3]). size mismatch for face_decoder_blocks.3.0.conv_block.0.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384 ]). size mismatch for face_decoder_blocks.3.0.conv_block.1.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([3 84]). size mismatch for face_decoder_blocks.3.0.conv_block.1.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384 ]). size mismatch for face_decoder_blocks.3.0.conv_block.1.running_mean: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.S ize([384]). size mismatch for face_decoder_blocks.3.0.conv_block.1.running_var: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Si ze([384]). size mismatch for face_decoder_blocks.3.1.conv_block.0.weight: copying a param with shape torch.Size([768, 768, 3, 3]) from checkpoint, the shape in current model is to rch.Size([384, 384, 3, 3]). size mismatch for face_decoder_blocks.3.1.conv_block.0.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384 ]). size mismatch for face_decoder_blocks.3.1.conv_block.1.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([3 84]). size mismatch for face_decoder_blocks.3.1.conv_block.1.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384 ]). size mismatch for face_decoder_blocks.3.1.conv_block.1.running_mean: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.S ize([384]). size mismatch for face_decoder_blocks.3.1.conv_block.1.running_var: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Si ze([384]). size mismatch for face_decoder_blocks.3.2.conv_block.0.weight: copying a param with shape torch.Size([768, 768, 3, 3]) from checkpoint, the shape in current model is to rch.Size([384, 384, 3, 3]). size mismatch for face_decoder_blocks.3.2.conv_block.0.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384 ]). size mismatch for face_decoder_blocks.3.2.conv_block.1.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([3 84]). size mismatch for face_decoder_blocks.3.2.conv_block.1.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384 ]). size mismatch for face_decoder_blocks.3.2.conv_block.1.running_mean: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.S ize([384]). size mismatch for face_decoder_blocks.3.2.conv_block.1.running_var: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Si ze([384]). size mismatch for face_decoder_blocks.4.0.conv_block.0.weight: copying a param with shape torch.Size([1024, 512, 3, 3]) from checkpoint, the shape in current model is t orch.Size([512, 256, 3, 3]). size mismatch for face_decoder_blocks.4.0.conv_block.0.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_decoder_blocks.4.0.conv_block.1.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([2 56]). size mismatch for face_decoder_blocks.4.0.conv_block.1.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_decoder_blocks.4.0.conv_block.1.running_mean: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.S ize([256]). size mismatch for face_decoder_blocks.4.0.conv_block.1.running_var: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Si ze([256]). size mismatch for face_decoder_blocks.4.1.conv_block.0.weight: copying a param with shape torch.Size([512, 512, 3, 3]) from checkpoint, the shape in current model is to rch.Size([256, 256, 3, 3]). size mismatch for face_decoder_blocks.4.1.conv_block.0.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_decoder_blocks.4.1.conv_block.1.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([2 56]). size mismatch for face_decoder_blocks.4.1.conv_block.1.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_decoder_blocks.4.1.conv_block.1.running_mean: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.S ize([256]). size mismatch for face_decoder_blocks.4.1.conv_block.1.running_var: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Si ze([256]). size mismatch for face_decoder_blocks.4.2.conv_block.0.weight: copying a param with shape torch.Size([512, 512, 3, 3]) from checkpoint, the shape in current model is to rch.Size([256, 256, 3, 3]). size mismatch for face_decoder_blocks.4.2.conv_block.0.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_decoder_blocks.4.2.conv_block.1.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([2 56]). size mismatch for face_decoder_blocks.4.2.conv_block.1.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256 ]). size mismatch for face_decoder_blocks.4.2.conv_block.1.running_mean: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.S ize([256]). size mismatch for face_decoder_blocks.4.2.conv_block.1.running_var: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Si ze([256]). size mismatch for face_decoder_blocks.5.0.conv_block.0.weight: copying a param with shape torch.Size([640, 256, 3, 3]) from checkpoint, the shape in current model is to rch.Size([320, 128, 3, 3]). size mismatch for face_decoder_blocks.5.0.conv_block.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128 ]). size mismatch for face_decoder_blocks.5.0.conv_block.1.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1 28]). size mismatch for face_decoder_blocks.5.0.conv_block.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128 ]). size mismatch for face_decoder_blocks.5.0.conv_block.1.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.S ize([128]). size mismatch for face_decoder_blocks.5.0.conv_block.1.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Si ze([128]). size mismatch for face_decoder_blocks.5.1.conv_block.0.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is to rch.Size([128, 128, 3, 3]). size mismatch for face_decoder_blocks.5.1.conv_block.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128 ]). size mismatch for face_decoder_blocks.5.1.conv_block.1.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1 28]). size mismatch for face_decoder_blocks.5.1.conv_block.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128 ]). size mismatch for face_decoder_blocks.5.1.conv_block.1.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.S ize([128]). size mismatch for face_decoder_blocks.5.1.conv_block.1.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Si ze([128]). size mismatch for face_decoder_blocks.5.2.conv_block.0.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is to rch.Size([128, 128, 3, 3]). size mismatch for face_decoder_blocks.5.2.conv_block.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128 ]). size mismatch for face_decoder_blocks.5.2.conv_block.1.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1 28]). size mismatch for face_decoder_blocks.5.2.conv_block.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128 ]). size mismatch for face_decoder_blocks.5.2.conv_block.1.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.S ize([128]). size mismatch for face_decoder_blocks.5.2.conv_block.1.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Si ze([128]). size mismatch for face_decoder_blocks.6.0.conv_block.0.weight: copying a param with shape torch.Size([320, 128, 3, 3]) from checkpoint, the shape in current model is to rch.Size([160, 64, 3, 3]). size mismatch for face_decoder_blocks.6.0.conv_block.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64] ). size mismatch for face_decoder_blocks.6.0.conv_block.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([6 4]). size mismatch for face_decoder_blocks.6.0.conv_block.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64] ). size mismatch for face_decoder_blocks.6.0.conv_block.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.S ize([64]). size mismatch for face_decoder_blocks.6.0.conv_block.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Si ze([64]). size mismatch for face_decoder_blocks.6.1.conv_block.0.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is to rch.Size([64, 64, 3, 3]). size mismatch for face_decoder_blocks.6.1.conv_block.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64] ). size mismatch for face_decoder_blocks.6.1.conv_block.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([6 4]). size mismatch for face_decoder_blocks.6.1.conv_block.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64] ). size mismatch for face_decoder_blocks.6.1.conv_block.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.S ize([64]). size mismatch for face_decoder_blocks.6.1.conv_block.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Si ze([64]). size mismatch for face_decoder_blocks.6.2.conv_block.0.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is to rch.Size([64, 64, 3, 3]). size mismatch for face_decoder_blocks.6.2.conv_block.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64] ). size mismatch for face_decoder_blocks.6.2.conv_block.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([6 4]). size mismatch for face_decoder_blocks.6.2.conv_block.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64] ). size mismatch for face_decoder_blocks.6.2.conv_block.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.S ize([64]). size mismatch for face_decoder_blocks.6.2.conv_block.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Si ze([64]). size mismatch for output_block.0.conv_block.0.weight: copying a param with shape torch.Size([16, 40, 3, 3]) from checkpoint, the shape in current model is torch.Size([3 2, 80, 3, 3]). size mismatch for output_block.0.conv_block.0.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for output_block.0.conv_block.1.weight: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for output_block.0.conv_block.1.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for output_block.0.conv_block.1.running_mean: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for output_block.0.conv_block.1.running_var: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for output_block.1.weight: copying a param with shape torch.Size([3, 16, 1, 1]) from checkpoint, the shape in current model is torch.Size([3, 32, 1, 1]). ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers