search

Jeff-sjtu / NIKI

Code of "NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation", CVPR 2023
254 stars 16 forks source link

About joints normalization(centerization) #26

Closed nnop closed 1 week ago

nnop commented 2 weeks ago

Hi, Jeff,

I'm a little confused how you normalize joints. It's not consistent in the codes. Some place use joints[0], some use mean joints[1,2]. image

In the following line of demo.py, the 3d joints are normalized against to center of joints[1,2]. https://github.com/Jeff-sjtu/NIKI/blob/69f94e7892fc815004a52efe7ce2fed2f71560aa/scripts/demo.py#L348

But, in the forward pass of FlowRegressor the joints is renormalized with respect to joints[0] which makes the previous normalization unnecessary. https://github.com/Jeff-sjtu/NIKI/blob/69f94e7892fc815004a52efe7ce2fed2f71560aa/niki/models/NIKI_1stage.py#L85

It seems most of the time you are using joints[0] for normalization. But sometimes use joints[1,2]'s center, e.g. in FlowRegressor.projection2uv(): https://github.com/Jeff-sjtu/NIKI/blob/69f94e7892fc815004a52efe7ce2fed2f71560aa/niki/models/NIKI_1stage.py#L247 The joints are normalized against to center of joints[1,2].

When should the joints being normalized against to joints[0] vs joints[1,2]'s center?

biansy000 commented 1 week ago

The default setting is to normalize against joints[0] (the pelvis joint). However, some video datasets do not provide the coordinates of the pelvis keypoint, so we have to normalize it against joints[1,2]'s center.