search

zycliao / skeleton-free-pose-transfer

[ECCV 2022] Skeleton-free Pose Transfer for Stylized 3D Characters
Other
193 stars 12 forks source link

reproduced visualization results #7

Closed twowwj closed 2 years ago

twowwj commented 2 years ago

Hi,

I followed the code for the division of the amass dataset. But found the visualization to be very poor. There were some bugs in the training code and I tried to fix them. image image Is there something I've missed? amass_splits = { 'train': ['BMLmovi', 'KIT', 'EKUT', 'TotalCapture', 'Eyes_Japan_Dataset', 'ACCAD', 'CMU'], 'test': ['HumanEva', 'SFU', 'Transitions_mocap', 'SSM_synced', 'DanceDB'], } Looking forward to your reply!

thx~

twowwj commented 2 years ago

https://github.com/zycliao/skeleton-free-pose-transfer/blob/ce04cac9d911131900e410dba8642bdee09c3f4b/train.py#L59 I set the "simplify=False" I suspect this is because I didn't simplify the amass data and kept 6890 vertices to train, could this be the main reason for the poor visualization results during training?

zycliao commented 2 years ago

I'm not sure what causes this. But simplify does not change the result much.

twowwj commented 2 years ago

image thanks for your reply! I noticed during training that loss_sw_sim is always 0; as well, loss_sw is moving from -0 to the negative half-axis for the first few epochs, gradually getting smaller and finally becoming 0. Is this normal?

zycliao commented 2 years ago

loss_sw_sim is used to make the skinning weight before and after augmentation to be the same. Without augmentation, it's always 0.

loss_sw contains two kl divergence and one is negative. So it could have any sign. This is explained in the paper in detail. When both KL div are zero, it means for sampled pairs of points, there's no error. This is normal when the sampled points' predicted skinning weights are rigid (like [1, 0, 0, 0])

twowwj commented 2 years ago

thanks for you reply!