Error with custom JSON keypoints extracted from openpose

[chuanenlin]

Hi, thank you for the interesting work and code.

I tried extracting skeleton keypoints from OpenPose (JSON files) then ran: python interpolate.py --model_path ./model/pretrained_full.pth -v1 ./examples/video1 -v2 ./examples/video2 -h1 720 -w1 720 -h2 720 -w2 720 -o ./outputs/interpolate-demo-custom.mp4 --keep_attr body --form matrix --nr_sample 5 --max_length 120

The following is the error I got: RuntimeError: invalid argument 4: Padding size should be less than the corresponding input dimension, but got: padding (3, 3) at dimension 2 of input [1 x 30 x 2] at c:\programdata\miniconda3\conda-bld\pytorch_1533090623466\work\aten\src\thcunn\generic/TemporalReflectionPadding.cu:32

Any idea why? Thanks!

[ChrisWu1997]

Hi, it seems that your input has only two frames ([1 x 30 x 2]). This is too short for the network which uses nn.ReflectionPad1d with pad size 3. nn.ReflectionPad1d requires input length to be longer than pad size. So it's better to use a longer video.

[chuanenlin]

Thanks for the reply! Using a video with more frames worked. May I also ask what performance cloning technique was used for the retargeting example? Would love to try out its performance with your great work. Thanks!

[ChrisWu1997]

For performance cloning, we use the method of this paper Deep Video-Based Performance Cloning. The code is not released because of some restrictions, but you can contact the author kfiraberman@gmail.com who can supply more detailed help.

And similar results can also be produced with the concurrent work of Everybody Dance Now, which has some unofficial implementations online.

[chuanenlin]

Thanks 🎉