search

Iven-Wu / CASA

code for Category-agnostic Skeletal Animal Reconstruction
Apache License 2.0
36 stars 2 forks source link

Questions regarding the re-animation #2

Closed cao-kais closed 1 year ago

cao-kais commented 1 year ago

Hi,

Thanks for the great work!

As described in the paper, the re-animation optimizes the joint angles of each bone for the skeletal shape, or the rigid transform for control point-based shape to minimize the Chamfer Distance between the reference shape and the predicted shape.

I was wondering how this is achieved for LASR and ViSER as these two methods rely on 2D observations (images, optical flow etc.) instead of directly optimizing to fit a target 3D shape. Could you provide more details on the procedure to do re-animation for LASR and ViSER in the paper? Thanks!

Iven-Wu commented 1 year ago

For re-animation we utilize 3D supervision, directly optimizing the chamfer distance between the predicted shape and reference ground-truth shape. We do not use any 2D observation in this setting. All three methods (CASA, LASR, ViSER) have the predictions of 3D shape at each iteration, which can be used for computing chamfer distances.

cao-kais commented 1 year ago

So during the optimization, the chamfer distance is used as the only loss function to fit the 3D shape? Just to confirm that my understanding is correct, this means that for LASR and ViSER, the canonical shape and blend skinning weights are fixed, while the control points are the only parameters being updated and optimized to fit the target 3D shape using chamfer distance as the loss?

Iven-Wu commented 1 year ago

Yes. The only difference is that for LASR/ViSER, the updated parameters are control points, while for CASA is the rotation angle for bones.

cao-kais commented 1 year ago

I see. Thank you so much for the clarification!