search

DenisTome / Lifting-from-the-Deep-release

Implementation of "Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image"
https://denistome.github.io/papers/lifting-from-the-deep
GNU General Public License v3.0
450 stars 133 forks source link

3D pose alignment #46

Closed matteorr closed 4 years ago

matteorr commented 5 years ago

Hi Denis, thanks for the great work.

Could you kindly share the aligned 3d poses or the code with which you perform the alignment?

Also, if I correctly understand how the alignment works, you are learning matrices e (of dimension J x 3L) and A (of dimension N x J), and N rotation matrices Ri (that are all about the y axis only), so that you can obtain every training set pose Pi by multiplying every element of the matrix e by the coefficients of a row ai of A, addind that to the mean pose of the training data and finally rotating it by Ri.

Once the optimization is done you can replace every training pose Pi with its `non-rotated' version (mu + ai * e) which basically points in the same direction of mu.

Could you please confirm my description is correct? If yes, could you explain why you need the noise variance sigma? If not, could you please help me understand?

Thanks a lot!

DenisTome commented 4 years ago

That is correct. sigma comes from probabilistic PCA which is a design decision we made then is reflected in Eq. 4.

If you look at the advantages of PPCA vs. PCA you can see why we chose that.