search

Hangz-nju-cuhk / Talking-Face-Generation-DAVS

Code for Talking Face Generation by Adversarially Disentangled Audio-Visual Representation (AAAI 2019)
MIT License
818 stars 173 forks source link

Error in loading Checkpoint file : Missing keys error #18

Open Gourimenon opened 5 years ago

Gourimenon commented 5 years ago

I faced some problems when I tried loading checkpoint file. The error I got is:

loading checkpoint '/home/gouriparvathymenon/Downloads/101_DAVS_checkpoint.pth' missing keys in state_dict: set(['module.block33.conv33_a_bn.num_batches_tracked', 'module.block12.conv12_b_bn.num_batches_tracked', 'module.block13.conv13_a_bn.num_batches_tracked', 'module.block32.conv32_a_bn.num_batches_tracked', 'module.block32.conv32_b_bn.num_batches_tracked', 'module.block34.conv34_a_bn.num_batches_tracked', 'module.block31.conv31_a_bn.num_batches_tracked', 'module.block12.conv12_a_bn.num_batches_tracked', 'module.block1.conv01_a_bn.num_batches_tracked', 'module.block33.conv33_b_bn.num_batches_tracked', 'module.block11.conv11_a_bn.num_batches_tracked', 'module.block42.conv42_b_bn.num_batches_tracked', 'module.block13.conv13_b_bn.num_batches_tracked', 'module.block24.conv24_b_bn.num_batches_tracked', 'module.block23.conv23_a_bn.num_batches_tracked', 'module.block25.conv25_b_bn.num_batches_tracked', 'module.block22.conv22_a_bn.num_batches_tracked', 'module.block34.conv34_b_bn.num_batches_tracked', 'module.block11.conv11_b_bn.num_batches_tracked', 'module.block21.conv21_a_bn.num_batches_tracked', 'module.block23.conv23_b_bn.num_batches_tracked', 'module.block1.conv01_b_bn.num_batches_tracked', 'module.block31.conv31_b_bn.num_batches_tracked', 'module.block42.conv42_a_bn.num_batches_tracked', 'module.block14.conv14_b_bn.num_batches_tracked', 'module.block26.conv26_a_bn.num_batches_tracked', 'module.block25.conv25_a_bn.num_batches_tracked', 'module.block22.conv22_b_bn.num_batches_tracked', 'module.block14.conv14_a_bn.num_batches_tracked', 'module.block26.conv26_b_bn.num_batches_tracked', 'module.block41.conv41_a_bn.num_batches_tracked', 'module.block21.conv21_b_bn.num_batches_tracked', 'module.block24.conv24_a_bn.num_batches_tracked', 'module.block41.conv41_b_bn.num_batches_tracked']) missing keys in state_dict: set(['module.convblock3.conv3_1_bn.num_batches_tracked', 'module.convblock5.conv5_0_bn.num_batches_tracked', 'module.convblock4.conv4_2_bn.num_batches_tracked', 'module.convblock2.conv2_0_bn.num_batches_tracked', 'module.convblock2.conv2_2_bn.num_batches_tracked', 'module.convblock3.conv3_3_bn.num_batches_tracked', 'module.convblock1.conv1_0_bn.num_batches_tracked', 'module.convblock1.conv1_1_bn.num_batches_tracked', 'module.convblock5.conv5_2_bn.num_batches_tracked', 'module.convblock3.conv3_0_bn.num_batches_tracked', 'module.convblock6.conv6_1_bn.num_batches_tracked', 'module.convblock3.conv3_2_bn.num_batches_tracked', 'module.deconv1_1_bn.num_batches_tracked', 'module.convblock4.conv4_0_bn.num_batches_tracked', 'module.convblock4.conv4_1_bn.num_batches_tracked', 'module.convblock2.conv2_1_bn.num_batches_tracked', 'module.conv7_1_bn.num_batches_tracked', 'module.convblock5.conv5_1_bn.num_batches_tracked', 'module.convblock4.conv4_3_bn.num_batches_tracked', 'module.convblock6.conv6_0_bn.num_batches_tracked']) missing keys in state_dict: set(['module.model1.bn3.num_batches_tracked', 'module.model1.bn1.num_batches_tracked', 'module.model1.bn2.num_batches_tracked', 'module.model1.bn5.num_batches_tracked', 'module.model2.bn1.num_batches_tracked', 'module.model2.bn2.num_batches_tracked']) missing keys in state_dict: set(['module.model.conv4.bn2.num_batches_tracked', 'module.model.top_m_0.bn2.num_batches_tracked', 'module.model.m0.b1_2.bn2.num_batches_tracked', 'module.model.m0.b1_4.bn2.num_batches_tracked', 'module.model.m0.b1_2.bn1.num_batches_tracked', 'module.model.conv3.bn2.num_batches_tracked', 'module.model.m0.b1_4.bn1.num_batches_tracked', 'module.model.m0.b1_3.bn1.num_batches_tracked', 'module.model.conv2.bn2.num_batches_tracked', 'module.model.m0.b3_2.bn2.num_batches_tracked', 'module.model.m0.b1_4.bn3.num_batches_tracked', 'module.model.conv2.bn1.num_batches_tracked', 'module.model.m0.b2_plus_1.bn2.num_batches_tracked', 'module.model.m0.b3_3.bn1.num_batches_tracked', 'module.model.m0.b2_3.bn2.num_batches_tracked', 'module.model.m0.b2_2.bn2.num_batches_tracked', 'module.model.m0.b1_1.bn3.num_batches_tracked', 'module.model.m0.b2_1.bn2.num_batches_tracked', 'module.model.conv2.bn3.num_batches_tracked', 'module.model.conv4.bn1.num_batches_tracked', 'module.model.m0.b2_4.bn2.num_batches_tracked', 'module.model.m0.b3_3.bn3.num_batches_tracked', 'module.model.conv4.bn3.num_batches_tracked', 'module.model.m0.b3_1.bn3.num_batches_tracked', 'module.model.m0.b2_2.bn3.num_batches_tracked', 'module.model.conv3.bn1.num_batches_tracked', 'module.model.m0.b3_4.bn2.num_batches_tracked', 'module.model.m0.b2_4.bn1.num_batches_tracked', 'module.model.m0.b3_2.bn1.num_batches_tracked', 'module.model.m0.b1_1.bn1.num_batches_tracked', 'module.model.m0.b1_1.bn2.num_batches_tracked', 'module.model.m0.b2_1.bn3.num_batches_tracked', 'module.model.top_m_0.bn1.num_batches_tracked', 'module.model.m0.b3_2.bn3.num_batches_tracked', 'module.model.m0.b1_2.bn3.num_batches_tracked', 'module.bn1.num_batches_tracked', 'module.model.conv2.downsample.0.num_batches_tracked', 'module.model.m0.b2_1.bn1.num_batches_tracked', 'module.model.m0.b3_1.bn1.num_batches_tracked', 'module.model.m0.b2_3.bn3.num_batches_tracked', 'module.model.bn1.num_batches_tracked', 'module.model.m0.b1_3.bn3.num_batches_tracked', 'module.model.bn_end0.num_batches_tracked', 'module.model.m0.b2_2.bn1.num_batches_tracked', 'module.model.m0.b3_1.bn2.num_batches_tracked', 'module.model.conv4.downsample.0.num_batches_tracked', 'module.model.m0.b3_4.bn1.num_batches_tracked', 'module.model.m0.b2_4.bn3.num_batches_tracked', 'module.model.m0.b2_plus_1.bn1.num_batches_tracked', 'module.model.m0.b2_3.bn1.num_batches_tracked', 'module.model.top_m_0.bn3.num_batches_tracked', 'module.model.m0.b3_3.bn2.num_batches_tracked', 'module.model.m0.b2_plus_1.bn3.num_batches_tracked', 'module.model.conv3.bn3.num_batches_tracked', 'module.model.m0.b3_4.bn3.num_batches_tracked', 'module.model.m0.b1_3.bn2.num_batches_tracked']) => loaded checkpoint '/home/gouriparvathymenon/Downloads/101_DAVS_checkpoint.pth' (step 21145000) Traceback (most recent call last): File "test_all.py", line 41, in for i2, data in enumerate(test_dataloader): File "/home/gouriparvathymenon/.local/share/virtualenvs/gouriparvathymenon-vLhDFwnm/lib/avatar/local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 637, in next return self._process_next_batch(batch) File "/home/gouriparvathymenon/.local/share/virtualenvs/gouriparvathymenon-vLhDFwnm/lib/avatar/local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch raise batch.exc_type(batch.exc_msg) KeyError: 'Traceback (most recent call last):\n File "/home/gouriparvathymenon/.local/share/virtualenvs/gouriparvathymenon-vLhDFwnm/lib/avatar/local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gouriparvathymenon/PycharmProjects/avatar/Talking-Face-Generation-DAVS/Dataloader/Test_load_audio.py", line 103, in getitem\n loader[\'A\'] = self.vid[\'A\']\nKeyError: \'A\'\n'

The command I used was this:

python test_all.py --test_root 001.wav --test_type audio --test_audio_video_length 99 --test_resume_path 101_DAVS_checkpoint.pth.tar

Can someone help me find where i went wrong? Thanks in advance.

Hangz-nju-cuhk commented 5 years ago

The error with "bn.num_batches_tracked" is because you are using pytorch version >0.4.0. where a new BN parameter is added. Problems could occur if you use pytorch version >0.2.0. However, I think this is not the reason for your program to stop. Anyway, even if there is no bug, our code cannot perform well on higher versions of pytorch at this moment.

Gourimenon commented 5 years ago

@Hangz-nju-cuhk Thank you for your reply. The issue was with the path to the video file. But unfortunately there is still some issues existing with audio.I would try what you suggested regarding the use of pytorch version< 0.4.0.

Ariel2013 commented 4 years ago

@Hangz-nju-cuhk sorry, i got the similar error. but i have re-version the torch to 0.2.0. so is there any suggestion?

fyi:

command: python test_all.py --test_root ./0572_0019_0003.wav --test_type audio --test_audio_video_length 99 --test_resume_path ./checkpoints/101_DAVS_checkpoint.pth.tar

$ pip list: torch 0.2.0.post1

and the errors:

Traceback (most recent call last):
  File "test_all.py", line 25, in <module>
    model = Gen_Model.GenModel(opt)
  File "/Users/xushenglai/Documents/tech/python/ai/Talking-Face-Generation-DAVS/Test_Gen_Models/Test_Audio_Model.py", line 31, in __init__
    self.ID_encoder = IdentityEncoder.IdentityEncoder(opt)
  File "/Users/xushenglai/Documents/tech/python/ai/Talking-Face-Generation-DAVS/network/IdentityEncoder.py", line 46, in __init__
    self.add_module('block' + str(01), BasicBlock(3, 32, name="01", conv_std=0.025253814, kernel_size=7, stride=2, padding=3))
  File "/Users/xushenglai/Documents/tech/python/ai/Talking-Face-Generation-DAVS/network/IdentityEncoder.py", line 21, in __init__
    self.initial()
  File "/Users/xushenglai/Documents/tech/python/ai/Talking-Face-Generation-DAVS/network/IdentityEncoder.py", line 34, in initial
    nn.init.normal(m.weight, std=self.conv_std)
AttributeError: 'module' object has no attribute 'init'

# xushenglai @ bogon in ~/Documents/tech/python/ai/Talking-Face-Generation-DAVS on git:master x [16:11:03] C:1
$ pip unstall

# xushenglai @ bogon in ~/Documents/tech/python/ai/Talking-Face-Generation-DAVS on git:master x [16:11:21] C:130
$ pip uninstall torch
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Uninstalling torch-0.1.12.post2:
  Would remove:
    /usr/local/lib/python2.7/site-packages/tools/*
    /usr/local/lib/python2.7/site-packages/torch-0.1.12.post2.dist-info/*
    /usr/local/lib/python2.7/site-packages/torch/*
  Would not remove (might be manually added):
    /usr/local/lib/python2.7/site-packages/torch/.serialization.py.swp
Proceed (y/n)? y
  Successfully uninstalled torch-0.1.12.post2

# xushenglai @ bogon in ~/Documents/tech/python/ai/Talking-Face-Generation-DAVS on git:master x [16:11:25]
$ pip innstal

# xushenglai @ bogon in ~/Documents/tech/python/ai/Talking-Face-Generation-DAVS on git:master x [16:11:43] C:130
$ pip install torch-0.2.0.post1-cp27-none-macosx_10_7_x86_64.whl
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Processing ./torch-0.2.0.post1-cp27-none-macosx_10_7_x86_64.whl
Requirement already satisfied: numpy in /usr/local/lib/python2.7/site-packages (from torch==0.2.0.post1) (1.16.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python2.7/site-packages (from torch==0.2.0.post1) (5.2)
Installing collected packages: torch
Successfully installed torch-0.2.0.post1

# xushenglai @ bogon in ~/Documents/tech/python/ai/Talking-Face-Generation-DAVS on git:master x [16:11:49]
$ python test_all.py --test_root ./0572_0019_0003.wav --test_type audio --test_audio_video_length 99 --test_resume_path ./checkpoints/101_DAVS_checkpoint.pth.tar
---------- Networks initialized -------------
=> loading checkpoint './checkpoints/101_DAVS_checkpoint.pth.tar'
missing keys in state_dict: set(['block42.conv42_a_bn.running_var', 'block26.conv26_b_bn.bias', 'block41.conv41_a_bn.running_var', 'block1.conv01_b.weight', 'block1.conv01_a_bn.weight', 'block11.conv11_a_bn.weight', 'block32.conv32_a.bias', 'block32.conv32_b_bn.running_mean', 'block26.conv26_b_bn.running_mean', 'block26.conv26_b.weight', 'block31.conv31_b.weight', 'block34.conv34_b.weight', 'block42.conv42_b_bn.weight', 'block12.conv12_b_bn.running_var', 'block42.conv42_a_bn.weight', 'block32.conv32_b_bn.running_var', 'block13.conv13_b.bias', 'block26.conv26_a_bn.weight', 'block25.conv25_a_bn.weight', 'block12.conv12_b.weight', 'block34.conv34_a_bn.running_var', 'block31.conv31_b_bn.weight', 'block13.conv13_a_bn.bias', 'block41.conv41_a_bn.bias', 'block24.conv24_b_bn.running_var', 'block14.conv14_a_bn.running_var', 'block24.conv24_a_bn.bias', 'block42.conv42_b.bias', 'block41.conv41_b.weight', 'block31.conv31_a_bn.running_var', 'block34.conv34_b_bn.running_mean', 'block13.conv13_a_bn.running_var', 'block11.conv11_b_bn.running_mean', 'block1.conv01_b_bn.running_mean', 'block13.conv13_a.weight', 'block34.conv34_a_bn.bias', 'block25.conv25_b_bn.running_mean', 'block11.conv11_b_bn.running_var', 'block24.conv24_b.weight', 'block33.conv33_a.weight', 'block33.conv33_b.bias', 'block42.conv42_b_bn.running_var', 'block21.conv21_a.weight', 'block33.conv33_b_bn.bias', 'block13.conv13_b.weight', 'block31.conv31_a_bn.weight', 'block12.conv12_a_bn.running_var', 'block12.conv12_a.weight', 'block14.conv14_b_bn.running_var', 'block21.conv21_a_bn.weight', 'block11.conv11_b_bn.weight', 'block22.conv22_b_bn.bias', 'block33.conv33_b_bn.running_var', 'block12.conv12_b_bn.weight', 'block23.conv23_a.bias', 'block24.conv24_a.weight', 'block11.conv11_b.weight', 'block42.conv42_a_bn.bias', 'block22.conv22_a.weight', 'block26.conv26_a_bn.bias', 'block11.conv11_b_bn.bias', 'block42.conv42_b.weight', 'block33.conv33_a_bn.bias', 'block22.conv22_a_bn.running_var', 'block25.conv25_a_bn.bias', 'block21.conv21_b_bn.running_mean', 'block14.conv14_b_bn.running_mean', 'block1.conv01_a_bn.bias', 'block21.conv21_b.weight', 'block1.conv01_b_bn.bias', 'block24.conv24_a_bn.weight', 'block12.conv12_a.bias', 'block24.conv24_b.bias', 'block31.conv31_a.weight', 'block13.conv13_b_bn.weight', 'block14.conv14_a.weight', 'block25.conv25_b_bn.running_var', 'block11.conv11_a_bn.bias', 'block14.conv14_a.bias', 'block24.conv24_a_bn.running_var', 'block13.conv13_a_bn.weight', 'block25.conv25_a.bias', 'block11.conv11_a_bn.running_var', 'block21.conv21_a_bn.running_mean', 'block32.conv32_b_bn.weight', 'block1.conv01_b_bn.running_var', 'block12.conv12_a_bn.bias', 'block25.conv25_a_bn.running_var', 'block32.conv32_a_bn.bias', 'block26.conv26_a_bn.running_mean', 'block31.conv31_b_bn.running_mean', 'block33.conv33_a_bn.weight', 'block13.conv13_b_bn.running_mean', 'block21.conv21_a_bn.bias', 'block31.conv31_a.bias', 'block24.conv24_a.bias', 'block41.conv41_b_bn.bias', 'block41.conv41_a.bias', 'block12.conv12_a_bn.running_mean', 'block32.conv32_b.bias', 'block32.conv32_b_bn.bias', 'block14.conv14_b.weight', 'block22.conv22_b.bias', 'block21.conv21_b_bn.running_var', 'block22.conv22_b_bn.running_mean', 'block42.conv42_a.weight', 'block21.conv21_a.bias', 'block34.conv34_a_bn.running_mean', 'block26.conv26_a.bias', 'block32.conv32_a_bn.weight', 'block21.conv21_a_bn.running_var', 'block31.conv31_b_bn.bias', 'block23.conv23_a_bn.running_mean', 'block22.conv22_a.bias', 'block25.conv25_b_bn.bias', 'block34.conv34_a.weight', 'block31.conv31_b.bias', 'block26.conv26_a.weight', 'block34.conv34_b_bn.bias', 'block23.conv23_a_bn.bias', 'block12.conv12_b.bias', 'block26.conv26_a_bn.running_var', 'block41.conv41_a_bn.running_mean', 'block25.conv25_a.weight', 'block14.conv14_b_bn.weight', 'block13.conv13_b_bn.bias', 'block12.conv12_b_bn.running_mean', 'block11.conv11_a_bn.running_mean', 'block41.conv41_b_bn.running_mean', 'block34.conv34_b_bn.weight', 'block23.conv23_a_bn.weight', 'block12.conv12_a_bn.weight', 'block13.conv13_a_bn.running_mean', 'block22.conv22_b.weight', 'block32.conv32_a_bn.running_mean', 'block25.conv25_b_bn.weight', 'block22.conv22_a_bn.bias', 'block14.conv14_a_bn.bias', 'block14.conv14_b_bn.bias', 'block23.conv23_b_bn.running_var', 'block11.conv11_b.bias', 'block13.conv13_b_bn.running_var', 'block41.conv41_b_bn.running_var', 'block23.conv23_b.bias', 'block1.conv01_a.bias', 'block25.conv25_b.weight', 'block14.conv14_a_bn.running_mean', 'block21.conv21_b_bn.bias', 'block22.conv22_a_bn.weight', 'block11.conv11_a.bias', 'block34.conv34_b_bn.running_var', 'block33.conv33_a_bn.running_var', 'block26.conv26_b_bn.running_var', 'block21.conv21_b_bn.weight', 'block34.conv34_a.bias', 'block42.conv42_a.bias', 'block25.conv25_b.bias', 'block24.conv24_b_bn.running_mean', 'block41.conv41_b_bn.weight', 'block33.conv33_a.bias', 'block32.conv32_b.weight', 'block23.conv23_b.weight', 'block22.conv22_a_bn.running_mean', 'block14.conv14_b.bias', 'block23.conv23_b_bn.running_mean', 'block25.conv25_a_bn.running_mean', 'block26.conv26_b_bn.weight', 'block12.conv12_b_bn.bias', 'block33.conv33_b_bn.weight', 'block33.conv33_a_bn.running_mean', 'block14.conv14_a_bn.weight', 'block22.conv22_b_bn.running_var', 'block42.conv42_a_bn.running_mean', 'block1.conv01_b_bn.weight', 'block42.conv42_b_bn.running_mean', 'block1.conv01_a.weight', 'block1.conv01_b.bias', 'block23.conv23_a_bn.running_var', 'block24.conv24_a_bn.running_mean', 'block1.conv01_a_bn.running_var', 'block23.conv23_b_bn.bias', 'block33.conv33_b.weight', 'block31.conv31_a_bn.running_mean', 'block41.conv41_b.bias', 'block33.conv33_b_bn.running_mean', 'block21.conv21_b.bias', 'block42.conv42_b_bn.bias', 'block34.conv34_a_bn.weight', 'block31.conv31_a_bn.bias', 'block34.conv34_b.bias', 'block24.conv24_b_bn.bias', 'block22.conv22_b_bn.weight', 'block11.conv11_a.weight', 'block24.conv24_b_bn.weight', 'block23.conv23_b_bn.weight', 'block32.conv32_a.weight', 'block13.conv13_a.bias', 'block32.conv32_a_bn.running_var', 'block1.conv01_a_bn.running_mean', 'block26.conv26_b.bias', 'block23.conv23_a.weight', 'block41.conv41_a_bn.weight', 'block31.conv31_b_bn.running_var', 'block41.conv41_a.weight'])
missing keys in state_dict: set(['convblock2.conv2_0_bn.weight', 'deconv1_1_new.bias', 'convblock6.conv6_0_bn.weight', 'conv7_2.weight', 'convblock3.conv3_2_bn.running_var', 'convblock5.conv5_0_bn.running_var', 'convblock5.conv5_0.weight', 'convblock1.conv1_1.weight', 'convblock5.conv5_1.bias', 'convblock2.conv2_2_bn.bias', 'convblock5.conv5_0_bn.bias', 'convblock5.conv5_1_bn.weight', 'convblock1.conv1_0_bn.bias', 'convblock1.conv1_1.bias', 'convblock3.conv3_3.weight', 'convblock5.conv5_0_bn.running_mean', 'convblock3.conv3_1.bias', 'convblock4.conv4_3_bn.running_mean', 'convblock1.conv1_0_bn.running_var', 'convblock4.conv4_0_bn.running_var', 'convblock3.conv3_1.weight', 'convblock6.conv6_0_bn.bias', 'convblock4.conv4_0_bn.bias', 'convblock5.conv5_1_bn.running_mean', 'convblock2.conv2_0.weight', 'convblock2.conv2_2_bn.running_var', 'convblock4.conv4_3.weight', 'convblock4.conv4_1_bn.bias', 'convblock4.conv4_2_bn.bias', 'convblock4.conv4_0.bias', 'convblock5.conv5_2.weight', 'convblock5.conv5_0.bias', 'convblock5.conv5_2_bn.bias', 'convblock2.conv2_2.weight', 'convblock4.conv4_1_bn.running_var', 'deconv1_1_bn.bias', 'convblock3.conv3_2_bn.bias', 'convblock4.conv4_3_bn.bias', 'convblock3.conv3_3.bias', 'convblock3.conv3_3_bn.weight', 'convblock6.conv6_0_bn.running_var', 'convblock1.conv1_0_bn.running_mean', 'conv7_1_bn.bias', 'convblock4.conv4_2_bn.running_var', 'convblock5.conv5_1_bn.bias', 'convblock4.conv4_3.bias', 'convblock6.conv6_1.weight', 'convblock2.conv2_1.bias', 'convblock3.conv3_0_bn.running_mean', 'convblock4.conv4_1.weight', 'convblock3.conv3_0_bn.running_var', 'convblock5.conv5_2.bias', 'convblock4.conv4_2.weight', 'convblock3.conv3_1_bn.running_mean', 'convblock4.conv4_3_bn.running_var', 'convblock2.conv2_1_bn.weight', 'convblock4.conv4_2.bias', 'convblock3.conv3_3_bn.bias', 'convblock4.conv4_1_bn.weight', 'deconv1_1_new.weight', 'convblock3.conv3_2.bias', 'convblock2.conv2_2.bias', 'convblock1.conv1_0.bias', 'convblock5.conv5_1.weight', 'convblock2.conv2_2_bn.weight', 'convblock3.conv3_1_bn.weight', 'convblock6.conv6_1_bn.bias', 'convblock5.conv5_2_bn.running_var', 'convblock5.conv5_1_bn.running_var', 'convblock1.conv1_1_bn.running_var', 'convblock6.conv6_1_bn.running_mean', 'convblock3.conv3_2_bn.running_mean', 'convblock4.conv4_0.weight', 'convblock2.conv2_1_bn.running_var', 'convblock6.conv6_0.weight', 'convblock1.conv1_1_bn.bias', 'conv7_1_bn.weight', 'convblock6.conv6_1_bn.running_var', 'convblock2.conv2_1_bn.bias', 'convblock5.conv5_2_bn.weight', 'convblock5.conv5_2_bn.running_mean', 'convblock3.conv3_2.weight', 'conv7_1.bias', 'convblock4.conv4_1.bias', 'convblock3.conv3_3_bn.running_mean', 'convblock3.conv3_2_bn.weight', 'convblock6.conv6_1_bn.weight', 'conv7_1.weight', 'convblock1.conv1_0.weight', 'convblock3.conv3_1_bn.running_var', 'convblock2.conv2_0_bn.bias', 'convblock4.conv4_0_bn.weight', 'convblock3.conv3_3_bn.running_var', 'deconv1_1_bn.running_var', 'convblock3.conv3_0.bias', 'convblock1.conv1_1_bn.running_mean', 'conv7_2.bias', 'deconv1_1_bn.running_mean', 'convblock4.conv4_2_bn.weight', 'convblock6.conv6_0_bn.running_mean', 'convblock6.conv6_0.bias', 'convblock3.conv3_1_bn.bias', 'convblock2.conv2_0_bn.running_mean', 'convblock1.conv1_1_bn.weight', 'convblock1.conv1_0_bn.weight', 'convblock4.conv4_2_bn.running_mean', 'deconv1_1_bn.weight', 'convblock2.conv2_1_bn.running_mean', 'convblock4.conv4_0_bn.running_mean', 'convblock3.conv3_0.weight', 'convblock4.conv4_3_bn.weight', 'convblock2.conv2_2_bn.running_mean', 'conv7_1_bn.running_var', 'convblock2.conv2_0.bias', 'convblock2.conv2_1.weight', 'convblock6.conv6_1.bias', 'convblock3.conv3_0_bn.bias', 'convblock5.conv5_0_bn.weight', 'convblock3.conv3_0_bn.weight', 'convblock4.conv4_1_bn.running_mean', 'conv7_1_bn.running_mean', 'convblock2.conv2_0_bn.running_var'])
missing keys in state_dict: set(['model2.bn2.running_mean', 'model2.bn2.running_var', 'model2.conv3.weight', 'model2.bn1.running_var', 'model2.conv1.weight', 'model2.bn1.running_mean', 'model1.bn2.weight', 'model2.bn2.weight', 'model1.bn1.bias', 'model1.bn3.weight', 'model1.conv4.weight', 'model1.conv1.weight', 'model1.conv2.weight', 'fc.weight', 'model1.bn3.bias', 'model1.bn3.running_var', 'model2.bn1.bias', 'fc.bias', 'model1.bn3.running_mean', 'model1.bn1.running_mean', 'model1.bn1.running_var', 'model1.bn2.running_var', 'model1.bn2.running_mean', 'model1.bn2.bias', 'model1.conv3.weight', 'model1.bn1.weight', 'model2.conv2.weight', 'model2.bn2.bias', 'model2.bn1.weight', 'model1.bn5.weight', 'model1.bn5.bias', 'model1.bn5.running_mean', 'model1.bn5.running_var'])
missing keys in state_dict: set(['model.m0.b1_2.bn2.weight', 'model.conv4.bn2.running_mean', 'model.conv3.conv3.weight', 'model.top_m_0.bn1.running_mean', 'model.m0.b2_3.bn1.bias', 'model.m0.b1_2.bn3.running_mean', 'model.conv3.bn3.weight', 'model.m0.b1_1.bn2.running_mean', 'model.m0.b2_4.bn2.running_var', 'model.m0.b1_1.bn2.bias', 'model.bn_end0.bias', 'model.m0.b2_plus_1.bn3.weight', 'model.m0.b2_4.bn3.weight', 'model.m0.b2_2.bn1.running_var', 'model.m0.b2_1.bn3.running_mean', 'model.conv4.bn3.weight', 'model.m0.b1_3.bn3.running_var', 'model.conv3.bn1.bias', 'model.m0.b3_3.bn1.running_mean', 'model.m0.b1_3.bn1.running_mean', 'model.m0.b2_plus_1.conv1.weight', 'model.conv3.conv2.weight', 'model.m0.b1_1.conv2.weight', 'model.m0.b1_4.bn1.running_mean', 'model.m0.b2_2.bn3.running_var', 'model.conv3.bn1.weight', 'model.m0.b2_3.bn1.running_var', 'model.conv2.bn1.running_var', 'model.m0.b2_2.bn2.running_var', 'model.m0.b2_4.bn1.running_mean', 'model.m0.b2_3.bn2.bias', 'model.conv3.bn3.running_mean', 'model.l0.weight', 'model.m0.b2_plus_1.bn3.running_var', 'model.m0.b2_plus_1.bn2.running_mean', 'model.m0.b1_4.bn1.running_var', 'fc.weight', 'model.m0.b2_4.bn3.bias', 'model.m0.b2_1.bn2.weight', 'model.m0.b3_4.bn1.running_var', 'model.conv4.conv2.weight', 'model.m0.b2_2.bn3.weight', 'bn1.weight', 'model.conv4.bn1.running_mean', 'model.m0.b2_1.conv3.weight', 'model.conv2.bn2.bias', 'model.m0.b3_4.conv3.weight', 'model.top_m_0.bn2.running_mean', 'model.m0.b1_1.bn3.bias', 'model.m0.b1_4.conv2.weight', 'model.m0.b3_4.bn2.weight', 'model.m0.b1_3.bn2.bias', 'model.conv2.bn2.running_var', 'model.m0.b2_plus_1.bn3.running_mean', 'model.m0.b2_1.bn2.running_mean', 'model.conv4.downsample.2.weight', 'model.conv2.conv2.weight', 'model.m0.b1_1.conv1.weight', 'model.conv4.conv3.weight', 'model.m0.b2_1.conv1.weight', 'model.m0.b2_4.bn2.bias', 'model.m0.b1_3.bn2.weight', 'model.m0.b3_1.conv3.weight', 'model.m0.b2_3.bn3.weight', 'model.m0.b2_2.bn1.bias', 'model.bn_end0.running_var', 'model.conv4.bn2.weight', 'model.m0.b1_4.bn2.weight', 'model.m0.b1_3.bn2.running_mean', 'model.m0.b2_2.bn2.bias', 'model.m0.b1_2.bn3.bias', 'model.m0.b1_2.bn3.running_var', 'model.conv3.bn1.running_mean', 'model.m0.b2_3.conv3.weight', 'model.m0.b1_4.bn2.bias', 'model.m0.b1_4.bn3.running_mean', 'model.m0.b3_4.conv2.weight', 'model.conv2.bn2.weight', 'model.conv3.conv1.weight', 'model.m0.b1_2.bn3.weight', 'model.conv_last0.weight', 'model.m0.b1_2.bn2.running_var', 'model.m0.b3_1.bn3.running_mean', 'model.m0.b2_plus_1.bn2.weight', 'model.conv2.downsample.2.weight', 'model.conv4.bn3.running_mean', 'model.m0.b1_4.bn2.running_var', 'model.conv2.bn1.weight', 'model.conv4.bn1.bias', 'model.m0.b3_2.conv2.weight', 'model.bn1.bias', 'model.conv2.downsample.0.running_var', 'model.conv2.bn3.bias', 'model.top_m_0.bn3.weight', 'model.m0.b1_3.bn2.running_var', 'model.conv4.downsample.0.running_var', 'bn1.bias', 'model.fc.weight', 'model.m0.b3_3.bn3.running_var', 'model.m0.b2_2.bn2.running_mean', 'model.m0.b2_1.bn3.bias', 'model.m0.b3_1.bn1.bias', 'model.m0.b2_3.bn3.bias', 'model.m0.b1_2.bn1.running_var', 'model.m0.b2_2.conv3.weight', 'model.m0.b3_3.bn2.weight', 'model.m0.b2_4.conv3.weight', 'model.m0.b3_4.bn2.bias', 'model.m0.b1_3.conv1.weight', 'model.m0.b2_1.bn3.running_var', 'model.m0.b1_2.bn2.bias', 'model.m0.b1_2.bn1.running_mean', 'model.m0.b2_2.bn2.weight', 'model.m0.b2_3.bn1.weight', 'model.bn1.running_var', 'model.m0.b2_4.bn2.running_mean', 'model.conv2.conv3.weight', 'model.conv3.bn2.running_mean', 'model.m0.b3_3.conv2.weight', 'model.top_m_0.bn3.bias', 'model.m0.b3_1.bn3.weight', 'model.m0.b1_1.bn1.running_mean', 'model.m0.b3_1.bn2.bias', 'model.m0.b1_4.bn2.running_mean', 'model.conv2.bn3.weight', 'model.m0.b2_3.bn2.weight', 'model.conv2.downsample.0.bias', 'model.conv4.downsample.0.bias', 'model.conv6.bias', 'model.m0.b2_4.bn1.running_var', 'model.m0.b2_1.bn2.running_var', 'model.m0.b1_2.bn1.bias', 'model.m0.b2_plus_1.bn2.bias', 'model.conv2.bn2.running_mean', 'model.m0.b2_3.bn2.running_mean', 'model.m0.b1_3.conv3.weight', 'model.conv4.bn1.running_var', 'model.m0.b3_4.bn3.running_var', 'model.top_m_0.bn1.weight', 'model.conv2.bn3.running_var', 'model.m0.b1_3.bn1.running_var', 'conv6.bias', 'model.conv4.bn3.running_var', 'model.m0.b3_1.bn2.running_mean', 'model.bn1.running_mean', 'model.m0.b2_plus_1.bn1.bias', 'model.m0.b3_4.bn1.weight', 'model.m0.b3_4.bn2.running_mean', 'model.m0.b1_4.bn1.bias', 'model.top_m_0.bn1.running_var', 'model.m0.b2_2.bn3.running_mean', 'model.m0.b3_3.bn2.running_var', 'model.m0.b3_3.bn1.running_var', 'model.m0.b2_4.conv1.weight', 'model.m0.b2_1.bn1.bias', 'model.m0.b3_4.conv1.weight', 'model.m0.b2_4.bn2.weight', 'model.m0.b1_1.bn2.running_var', 'model.top_m_0.conv3.weight', 'model.conv2.bn1.running_mean', 'model.conv3.bn2.weight', 'fc.bias', 'model.m0.b1_3.bn1.bias', 'model.m0.b3_2.bn1.weight', 'model.m0.b3_1.bn1.running_mean', 'model.m0.b3_3.bn2.running_mean', 'model.m0.b2_1.bn1.running_var', 'model.m0.b3_2.bn2.weight', 'model.conv2.bn1.bias', 'model.m0.b3_1.bn1.running_var', 'model.top_m_0.bn2.weight', 'model.m0.b3_1.bn2.running_var', 'model.m0.b1_4.conv1.weight', 'model.m0.b3_3.conv3.weight', 'model.m0.b1_1.bn2.weight', 'model.m0.b1_2.conv2.weight', 'model.m0.b1_4.bn1.weight', 'model.m0.b1_4.bn3.weight', 'model.top_m_0.conv2.weight', 'model.m0.b1_3.bn3.weight', 'model.m0.b2_2.bn1.weight', 'model.top_m_0.bn3.running_var', 'model.top_m_0.bn1.bias', 'model.conv3.bn2.bias', 'model.m0.b2_2.conv1.weight', 'model.top_m_0.bn2.running_var', 'model.m0.b3_1.conv2.weight', 'model.m0.b3_1.conv1.weight', 'model.m0.b2_4.conv2.weight', 'model.m0.b3_2.bn1.bias', 'model.m0.b3_2.bn3.running_var', 'model.m0.b2_plus_1.conv3.weight', 'bn1.running_var', 'model.fc.bias', 'model.m0.b2_3.bn3.running_var', 'model.m0.b1_4.bn3.running_var', 'model.m0.b2_plus_1.bn3.bias', 'model.m0.b1_1.bn3.running_mean', 'model.m0.b3_3.bn1.weight', 'model.m0.b2_3.conv1.weight', 'model.m0.b2_2.bn1.running_mean', 'model.m0.b3_4.bn3.running_mean', 'model.conv4.bn2.running_var', 'model.m0.b3_2.bn3.running_mean', 'model.m0.b1_2.bn1.weight', 'model.m0.b3_1.bn1.weight', 'model.m0.b3_2.bn2.running_mean', 'model.m0.b2_1.conv2.weight', 'model.m0.b3_1.bn3.bias', 'model.m0.b1_3.bn1.weight', 'model.m0.b2_2.bn3.bias', 'model.m0.b2_plus_1.conv2.weight', 'model.m0.b3_2.bn2.bias', 'model.top_m_0.bn2.bias', 'model.m0.b2_plus_1.bn1.weight', 'model.conv1.weight', 'bn1.running_mean', 'model.conv5.weight', 'model.conv4.downsample.0.running_mean', 'model.m0.b2_3.bn2.running_var', 'model.m0.b1_2.conv1.weight', 'model.m0.b3_2.bn2.running_var', 'model.m0.b1_4.conv3.weight', 'model.conv3.bn3.bias', 'model.m0.b2_4.bn3.running_mean', 'model.m0.b2_3.bn3.running_mean', 'model.m0.b3_1.bn3.running_var', 'model.m0.b3_4.bn1.running_mean', 'model.m0.b1_1.bn1.running_var', 'model.conv4.bn3.bias', 'model.conv4.conv1.weight', 'model.m0.b2_plus_1.bn1.running_mean', 'model.m0.b1_1.conv3.weight', 'model.conv3.bn3.running_var', 'model.conv4.bn1.weight', 'model.m0.b1_3.bn3.running_mean', 'model.m0.b2_4.bn3.running_var', 'model.conv4.downsample.0.weight', 'model.m0.b2_1.bn3.weight', 'model.conv4.bn2.bias', 'model.conv2.conv1.weight', 'model.m0.b3_3.bn3.running_mean', 'model.m0.b1_4.bn3.bias', 'model.m0.b1_3.bn3.bias', 'model.conv_last0.bias', 'model.m0.b3_4.bn3.bias', 'model.m0.b1_1.bn3.weight', 'model.conv2.bn3.running_mean', 'model.m0.b2_1.bn1.running_mean', 'model.m0.b2_4.bn1.bias', 'model.top_m_0.bn3.running_mean', 'model.m0.b3_2.bn3.weight', 'model.bn_end0.weight', 'conv6.weight', 'model.conv1.bias', 'model.conv2.downsample.0.weight', 'model.conv3.bn2.running_var', 'model.m0.b1_3.conv2.weight', 'model.m0.b3_3.bn2.bias', 'model.conv5.bias', 'model.m0.b3_4.bn2.running_var', 'model.m0.b1_2.conv3.weight', 'model.m0.b3_2.conv3.weight', 'model.m0.b2_1.bn2.bias', 'model.m0.b2_1.bn1.weight', 'model.m0.b2_4.bn1.weight', 'model.m0.b3_2.bn1.running_mean', 'model.m0.b2_plus_1.bn1.running_var', 'model.m0.b2_plus_1.bn2.running_var', 'model.m0.b3_2.bn3.bias', 'model.bn1.weight', 'model.conv3.bn1.running_var', 'model.m0.b3_2.bn1.running_var', 'model.m0.b3_3.conv1.weight', 'model.m0.b3_3.bn3.bias', 'model.m0.b3_4.bn3.weight', 'model.m0.b3_4.bn1.bias', 'model.m0.b3_2.conv1.weight', 'model.m0.b2_3.conv2.weight', 'model.m0.b1_1.bn3.running_var', 'model.m0.b1_1.bn1.bias', 'model.m0.b3_3.bn3.weight', 'model.conv6.weight', 'model.m0.b3_3.bn1.bias', 'model.m0.b1_2.bn2.running_mean', 'model.m0.b1_1.bn1.weight', 'model.top_m_0.conv1.weight', 'model.m0.b2_2.conv2.weight', 'model.l0.bias', 'model.m0.b3_1.bn2.weight', 'model.conv2.downsample.0.running_mean', 'model.bn_end0.running_mean', 'model.m0.b2_3.bn1.running_mean'])
=> loaded checkpoint './checkpoints/101_DAVS_checkpoint.pth.tar' (step 21145000)
Traceback (most recent call last):
  File "test_all.py", line 41, in <module>
    for i2, data in enumerate(test_dataloader):
  File "/usr/local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 201, in __next__
    return self._process_next_batch(batch)
  File "/usr/local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 221, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n  File "/usr/local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 40, in _worker_loop\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/Users/xushenglai/Documents/tech/python/ai/Talking-Face-Generation-DAVS/Dataloader/Test_load_audio.py", line 103, in __getitem__\n    loader[\'A\'] = self.vid[\'A\']\nKeyError: \'A\'\n'

thanks in advance.

duguiming111 commented 4 years ago

image

yxt132 commented 4 years ago

i have the same problem but have not found a solution. @Hangz-nju-cuhk can you update the repo to support higher version of pytorch?

Hangz-nju-cuhk commented 4 years ago

@yxt132 Yes I plan to do so within 3 months XD. But please do not have too much hope in me, as there are also other ongoing projects >_<.