Unclear on how to run motion transfer

wanshun123 commented 5 years ago

I am trying to transfer the facial expressions in one gif (driving_video.gif) to another photo (source_image.jpg), using the pretrained checkpoint provided. In the readme the following command is given (missing a file - presumably this should be python demo.py):

python --config config/moving-gif.yaml --driving_video sup-mat/driving_video.gif --source_image sup-mat/source_image.gif --checkpoint path/to/checkpoint

I've tried with the following (I have just put driving_video.gif, source_image.jpg and moving-gif-ckp.pth.tar in the root of the project folder):

python demo.py --config config/moving-gif.yaml --driving_video driving_video.gif --source_image source_image.jpg --checkpoint moving-gif-ckp.pth.tar

This results in the following:

Traceback (most recent call last):
  File "demo.py", line 52, in <module>
    Logger.load_cpk(opt.checkpoint, generator=generator, kp_detector=kp_detector)
  File "/home/paperspace/monkey/monkey/logger.py", line 54, in load_cpk
    generator.load_state_dict(checkpoint['generator'])
  File "/home/paperspace/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for MotionTransferGenerator:
        Unexpected key(s) in state_dict: "appearance_encoder.down_blocks.0.norm.num_batches_tracked", "appearance_encoder.down_blocks.1.norm.num_batches_tracked", "appearance_encoder.down_blocks.2.norm.num_batches_tracked", "appearance_encoder.down_blocks.3.norm.num_batches_tracked", "appearance_encoder.down_blocks.4.norm.num_batches_tracked", "appearance_encoder.down_blocks.5.norm.num_batches_tracked", "dense_motion_module.group_blocks.0.norm.num_batches_tracked", "dense_motion_module.group_blocks.1.norm.num_batches_tracked", "dense_motion_module.hourglass.encoder.down_blocks.0.norm.num_batches_tracked", "dense_motion_module.hourglass.encoder.down_blocks.1.norm.num_batches_tracked", "dense_motion_module.hourglass.encoder.down_blocks.2.norm.num_batches_tracked", "dense_motion_module.hourglass.encoder.down_blocks.3.norm.num_batches_tracked", "dense_motion_module.hourglass.encoder.down_blocks.4.norm.num_batches_tracked", "dense_motion_module.hourglass.decoder.up_blocks.0.norm.num_batches_tracked", "dense_motion_module.hourglass.decoder.up_blocks.1.norm.num_batches_tracked", "dense_motion_module.hourglass.decoder.up_blocks.2.norm.num_batches_tracked", "dense_motion_module.hourglass.decoder.up_blocks.3.norm.num_batches_tracked", "dense_motion_module.hourglass.decoder.up_blocks.4.norm.num_batches_tracked", "video_decoder.up_blocks.0.norm.num_batches_tracked", "video_decoder.up_blocks.1.norm.num_batches_tracked", "video_decoder.up_blocks.2.norm.num_batches_tracked", "video_decoder.up_blocks.3.norm.num_batches_tracked", "video_decoder.up_blocks.4.norm.num_batches_tracked", "video_decoder.up_blocks.5.norm.num_batches_tracked", "refinement_module.r0.norm1.num_batches_tracked", "refinement_module.r0.norm2.num_batches_tracked", "refinement_module.r1.norm1.num_batches_tracked", "refinement_module.r1.norm2.num_batches_tracked", "refinement_module.r2.norm1.num_batches_tracked", "refinement_module.r2.norm2.num_batches_tracked", "refinement_module.r3.norm1.num_batches_tracked", "refinement_module.r3.norm2.num_batches_tracked".

AliaksandrSiarohin commented 5 years ago

What is the version of pytorch you are using? Maybe this is the reason https://discuss.pytorch.org/t/solved-unexpected-key-s-in-state-dict-batches-tracked/23884?

wanshun123 commented 5 years ago

0.4.0 (verified by python -c "import torch; print(torch.__version__)")

AliaksandrSiarohin commented 5 years ago

I train in 0.4.1, never tried 0.4.0. In the issue they have the same problem, upgrade solve the issue.

wanshun123 commented 5 years ago

Pytorch needing to be 0.4.1 was the issue.

AliaksandrSiarohin / monkey-net

Unclear on how to run motion transfer #4