search

DeepMotionEditing / deep-motion-editing

An end-to-end library for editing and rendering motion of 3D characters with deep learning [SIGGRAPH 2020]
BSD 2-Clause "Simplified" License
1.58k stars 256 forks source link

About Intra-Structure Training On Custom dataset #162

Closed FishWoWater closed 3 years ago

FishWoWater commented 3 years ago

Hi! Thanks for your nice work!

I have a question when considering intra-structure training setting on my custom dataset. The encoder learns latent code from offsets and rotations of model A, and then given offsets of model B, the encoder is expected to output rotations conditioned on those offsets, right? I guess bone proportion difference is addressed in this way, but I am wondering whether such practice is robust to different initial poses(e.g. model A and model B has different initial poses). And if I have limited number of characters (less than 10 or even less than 5), I am wondering whether the generalization ability to different initial pose will be affected.

Looking forward to your reply, thanks! @kfiraberman @PeizhuoLi

PeizhuoLi commented 3 years ago

Hi, thank you for your interest in our project.

You are right. Generally we expect the initial pose to be in similar T-pose. Non T-pose input could lead to failure, especially when there are only a few training characters. I would recommend you to retarget the rest pose in T-pose before using our code.

FishWoWater commented 3 years ago

OK, thanks!