search

akanazawa / hmr

Project page for End-to-end Recovery of Human Shape and Pose
Other
1.54k stars 395 forks source link

Why is the silhoutte inclined? #71

Closed saturosfz closed 5 years ago

saturosfz commented 5 years ago

Dear author, thank you for your great work. I downloaded the tfrecords, and plot the 3D points of SMPL model. I directly use the SMPL vertices without transforms. The plot seems great from the front view, but the sihoutte is inclined. bbbb 000000 5 The first is the tilted silhoutte, and the second is the corresponding image. From the image I think the man is standing straight. Is there something wrong with my plot? Thank you!

akanazawa commented 5 years ago

Hi,

This could be two things:

  1. Ambiguity, the model is cheating to get the right hight by slightly tilting people like this. It's not explicitly recovering the floor, so things like this can happen.

  2. The global coordinate frame is slightly off, in this world, where that feet is might be the floor and if you applied that transformation it would be up right. You could double check this with the training data, see if it looks up right from side view. I think it's a problem of both, when you visualize the ground truth H36M pose I remember it still looks somewhat tilted..

Best,

Angjoo

neonb88 commented 5 years ago

Dear Ms. Kanazawa,

Our team has noticed many cases of this; HMR captures the pose well, but the resulting SMPL model is often leaning forward. Is this a result of training data also being taken leaning forward?

Thanks!

akanazawa commented 5 years ago

Yes there are two issues that can lead to, one due to depth ambiguity (hard to resolve) second due to the coordinate space of the training data. I think it's a mixture of the two. There is inherent ambiguity that makes this problem difficult.