search

facebookresearch / frankmocap

A Strong and Easy-to-use Single View 3D Hand+Body Pose Estimator
Other
2.14k stars 374 forks source link

problems about perspective projection #141

Closed biansy000 closed 3 years ago

biansy000 commented 3 years ago

I find it a little confused to understand the projecting methods used in the code. Especially, I cannot understand convert_smpl_to_bbox and convert_bbox_to_oriIm functions in copy_and_test.py. It seems that you are not using conventional perspective projection (as used in SPIN model), and the z-axis coordinate is not used in projection procedure. The use of resnet_input_size_half is also strange

Could you please explain briefly about how your projection method works?

penincillin commented 3 years ago

@biansy000 We use orthogonal projection for both body and hand, which directly taking the x, y coords and neglects the z coords. SPIN also uses orthogonal projection instead of perspective projection. This practice (using orthogonal projection) can be traced back to HMR. The reason is that we assume the agnostic of camera intrinsics, which cannot be faithfully estimated from in-the-wild images. Therefore, we ease the problem and choose to use orthogonal projection.

biansy000 commented 3 years ago

Thanks! I understand the code now. BTW, I think that SPIN and HMR at least in some part of the code do use perspective projection (https://github.com/nkolot/SPIN/blob/master/utils/geometry.py) :joy:. That is why I felt confused.