search

AliaksandrSiarohin / monkey-net

Animating Arbitrary Objects via Deep Motion Transfer
467 stars 81 forks source link

Transfering motion to a bird. #15

Closed TsainGra closed 4 years ago

TsainGra commented 4 years ago

Hello, I am trying to transfer motion of one bird to another. However, due to lack of data, I am trying to infer from using moving-gif weights provided with the code. After adding a white background to all frames, I tried to reconstruct the video giving the first frame of the driving video as the source image. I observed that apart from the generator not able to generate convincing images, sometimes the key points were estimated incorrectly. Is there any sort of relation in which the keypoint detector depends on the object (mostly quadrupeds in moving-gif) the network was trained on.

AliaksandrSiarohin commented 4 years ago

Yes, the keypoints is trained along with entire system for specific dataset. So in case of moving gif keypoint detector is specialized on 4 leged cartonish aninals. So it is not surprising that it not working on birds. Depends on how much data you have you can try to finetune the system on birds.