Nicholasli1995 / EvoSkeleton

Official project website for the CVPR 2020 paper (Oral Presentation) "Cascaded deep monocular 3D human pose estimation wth evolutionary training data"
https://arxiv.org/abs/2006.07778
MIT License
333 stars 44 forks source link

About the coordinate system #32

Closed NoLookDefense closed 3 years ago

NoLookDefense commented 3 years ago

Hi. I want to consult something about the coordinate system. I saw that before evolution, there is a function called to_local(), which "represents the bone vectors in the local coordinate systems" https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L285 I feel confused about this part. Will the coordinate be converted from camera coordinate to local coordinate? And what's definition of local coordinate and what's the meaning of the conversion? BTW, it will be my honour if you are willing to provide me with some references. I am looking forward ton your reply.

Nicholasli1995 commented 3 years ago

Hi. I want to consult something about the coordinate system. I saw that before evolution, there is a function called to_local(), which "represents the bone vectors in the local coordinate systems" https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L285

I feel confused about this part. Will the coordinate be converted from camera coordinate to local coordinate? And what's definition of local coordinate and what's the meaning of the conversion? BTW, it will be my honour if you are willing to provide me with some references. I am looking forward ton your reply.

Hi, each bone vector is represented in a local coodinate system. A local system is computed based on a 3D skeleton. The conversion is done with the inner product. The meaning of the conversion is: when the 3D pose is expressed in local systems, it is not affected by global rigid transformation. This makes it easier to represent a posture with joint angles.

You can take a look at section 3.1 of the paper, as well as section 7 of the supplementary material (https://arxiv.org/pdf/2006.07778.pdf)

NoLookDefense commented 3 years ago

Thank you. I found that the angle of upper limb is limited by angle_spread https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L20 , and the angle of lower limb is constrained by sepPlane https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L22

But I feel confused about the sepPlane. As shown in the figure, it contains a list of a length 11. Some of them are empty and some of them contain a numpy array in the shape of (121, 61, 4) . In each numpy array, it has some values between [-1, 1], or nan.

I feel extremely confused about this part. The size (121, 61) may mean the quantised result of spherical coordinates, but what about the third dimension 4? Also, what's the physical meaning of the values in each numpy array (like the nan)?

I am looking forward to your reply.

Screenshot 2021-05-30 at 15 05 11
Nicholasli1995 commented 3 years ago

Thank you. I found that the angle of upper limb is limited by angle_spread https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L20

, and the angle of lower limb is constrained by sepPlane https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L22

But I feel confused about the sepPlane. As shown in the figure, it contains a list of a length 11. Some of them are empty and some of them contain a numpy array in the shape of (121, 61, 4) . In each numpy array, it has some values between [-1, 1], or nan.

I feel extremely confused about this part. The size (121, 61) may mean the quantised result of spherical coordinates, but what about the third dimension 4? Also, what's the physical meaning of the values in each numpy array (like the nan)?

I am looking forward to your reply.

Screenshot 2021-05-30 at 15 05 11

That is the vector used to define the separate plane. Conditioned on the parent bone orientation, this vector defines which half-sphere is valid. After projecting the bone vector to the separate plane, there is a bounding box as the valid region. The usage is shown here: https://github.com/Nicholasli1995/EvoSkeleton/blob/f31e7c2e453cfb01cbb343c71f3d94dcf98efc4f/libs/skeleton/anglelimits.py#L105

See Figure 3 and Sec 3.2 of the reference: https://openaccess.thecvf.com/content_cvpr_2015/papers/Akhter_Pose-Conditioned_Joint_Angle_2015_CVPR_paper.pdf for more details.

Nicholasli1995 commented 3 years ago

For future Q&A, please use the discussion function here: https://github.com/Nicholasli1995/EvoSkeleton/discussions. Issues are only used for bug reporting and enhancement.