search

snap-research / articulated-animation

Code for Motion Representations for Articulated Animation paper
https://snap-research.github.io/articulated-animation/
Other
1.21k stars 348 forks source link

Does it work with cats? #11

Open franz101 opened 3 years ago

franz101 commented 3 years ago

Really important. Just asking for science.

AliaksandrSiarohin commented 3 years ago

The problem is that there are no dataset of cats, that could be used for training this staff. If there were at least 100 cat videos similar to one in voxceleb it will be worth to try. Without retraining it has no idea of how to open the cat mount and close cat eyes. ![01](https://user-images.githubuser

https://user-images.githubusercontent.com/3857997/116883860-bdef9400-ac2e-11eb-9e37-b88e16176d40.mp4

content.com/3857997/116883856-bcbe6700-ac2e-11eb-83b9-e0e8b3c5662b.jpeg)

https://user-images.githubusercontent.com/3857997/116883857-bd56fd80-ac2e-11eb-9363-7043edcf113b.mp4

RobertLowe commented 3 years ago

I find this vital to the project.

hetaoaoao commented 3 years ago

The problem is that there are no dataset of cats, that could be used for training this staff. If there were at least 100 cat videos similar to one in voxceleb it will be worth to try. Without retraining it has no idea of how to open the cat mount and close cat eyes. ![01](https://user-images.githubuser

result.mp4 content.com/3857997/116883856-bcbe6700-ac2e-11eb-83b9-e0e8b3c5662b.jpeg)

04.mp4

Amazing work~ What could be the proper way to get fine results if we already have 100+ cats video in hand? Train from scratch or do some sort of fine tuning based on exsiting vox model?

AliaksandrSiarohin commented 3 years ago

Finetuning.

hetaoaoao commented 3 years ago

Finetuning.

Thanks. What if I need cat/dog talks like human, any suggestions? Will it do the trick by combining dog yelling videos and human talking videos as training data?

AliaksandrSiarohin commented 3 years ago

This could happen in principle. But need to be tested out.

hacker009-sudo commented 8 months ago

File "/content/articulated-animation/logger.py", line 80, in load_cpk optimizer_reconstruction.load_state_dict(checkpoint['optimizer_reconstruction']) File "/usr/local/lib/python3.7/site-packages/torch/optim/optimizer.py", line 116, in load_state_dict raise ValueError("loaded state dict contains a parameter group " ValueError: loaded state dict contains a parameter group that doesn't match the size of optimizer's group

@AliaksandrSiarohin @hetaoaoao @franz101 @sergeytulyakov @RobertLowe , I am getting above error while fine - tuning please guide how to resolve the issue.