search

snap-research / articulated-animation

Code for Motion Representations for Articulated Animation paper
https://snap-research.github.io/articulated-animation/
Other
1.21k stars 348 forks source link

Question about the pixel-wise flow predictor #16

Open silverbulletmdc opened 3 years ago

silverbulletmdc commented 3 years ago

Hello. I have read your paper and your code carefully. I think this is a very high-quality job. But when I train the model, I found that some regions are not oval (like the hair in TED, or the eyes in VOX). Here are some examples. image image

I check the implementation of the flow predictor. It generates the sparse motion by rendering a gaussian heatmap according to the region params. In my experience, the Gaussian distribution generates elliptical regions. Considering some regions are not oval, can the sparse motion accurate enough? Or is there some other mechanism to handle this situation? Do you think this could be a direction for improvement in the future?

silverbulletmdc commented 3 years ago

The weird thing is that when I retrain the model using the same configuration, the two eyes are learned on two different oval regions. image