search

steb6 / ISBFSAR

Interactive Skeleton Based Few Shot Action Recognition
14 stars 2 forks source link

smpl pose ? #3

Closed luoww1992 closed 1 year ago

luoww1992 commented 2 years ago

i have run it with only hpe.py, and i notice the smpl in the code , so, Q1: the results in hpe.py are 3d keypoints and the skeleton length and human box ? Q2: i want to get the smpl pose for the results , how to do ? i find it shows smpl image in project metrabs . I'm checking the project metrabs code , and can you give me some advices or some python module to get it ?

steb6 commented 2 years ago

Q1: The results of HumanPoseEstimator.estimate are the absolute pose of the predicted 3d skeleton (center w.r.t. first entry to obtain 3d root pose), a list of edges (each entry represent a "bone", so it indicates that the 2 points corresponding to these entries should be connected with a line) and the bounding box of the human. Q2: Once that you have the 30 points it should not be difficult to convert it into an smpl since the format is correct.

luoww1992 commented 2 years ago

thanks, i just want to show the result 3p pose in UE, but it seem to get the 3d smpl pose. if we can use the 3d keypoints ,directly， that will be very good .

steb6 commented 2 years ago

What you mean with UE? You can convert the skeleton into a lot of skeleton types, check hpe.py for details.

luoww1992 commented 2 years ago

i mean is that i want to use the result pose to drive the human skeleton in UE4， i don't know how to drive the human skeleton by 3d pose keypoints, so i need to change the 3d keypoints to the smpl Joint rotations or euler angle or quaternion. and, i can change smpl+head_30 to smpl_24, but the result is 3d keypoints, not smpl Joint rotations or euler angle with parent-child relationship. maybe i need to perform inverse kinematics to compute rotations. if you know some methods, can you share me some advices ?

steb6 commented 2 years ago

I don't even know if it is possible to do that inverse kinematics: you cant compute the roll for most of the joints. Take a look at this repo, I think that it could work better for you.

luoww1992 commented 2 years ago

@StefanoBerti i notice there are some agrs need to add to the inputs in project, like: intrinsic_matrix, extrinsic_matrix, world_up_vector...., how to use these in code ? Add these before creating engine ?

steb6 commented 2 years ago

The engines are independent from those variables. You should find everything you need inside utils/params.py and they should be imported automatically when you start main.py or modules/hpe/hpe.py.

luoww1992 commented 2 years ago

@StefanoBerti i have see the params.py , but there have a part of args only, the others ? i think the args (intrinsic_matrix, extrinsic_matrix, world_up_vector) are important when we want to get a better result。

steb6 commented 2 years ago

I think you are interested in the RealSenseIntrinsics class in params.py, that are used for the homography. By modifying its values, you should be able to use different intrinsics . For the world_up_vector you may just need to invert the output pose along the second dimension and for the extrinsic you just need to add the camera position and rotation to the output pose.