How to correctly get the 3D Joints position

[vittorione94]

Hello,

I'm trying to extract the 3D joints positions using PHALP and 3D Human Body Prior (link), to later display them in a physics engine.

This is a sample code of how I'm doing this for just a single frame.

from human_body_prior.tools.rotation_tools import matrot2aa
from human_body_prior.body_model.body_model import BodyModel
from human_body_prior.tools.omni_tools import copy2cpu as c2c

results = joblib.load(file_path)
bm_fname = "body_models/smplh/neutral/model.npz"
k = 0 
body_pose = results[k]['smpl'][0]['body_pose']
global_orient_euler = matrot2aa(torch.Tensor(results[k]['smpl'][0]['global_orient']).to(comp_device))
eulers = matrot2aa(torch.Tensor(body_pose[:-2,::]).to(comp_device))

num_betas = 10  # number of body parameters
body_parms = {
        'pose_body': 
            torch.unsqueeze(eulers.flatten(), axis=0), # controls the body
        'betas': 
            torch.Tensor(np.repeat(results[k]['smpl'][0]['betas'][:num_betas][np.newaxis], repeats=1, axis=0)).to(comp_device), # controls the body shape. Body shape is static
        'root_orient': 
            torch.unsqueeze(global_orient_euler.flatten(),axis=0)
        }

body_pose = BodyModel(bm_fname)(**{
                k: torch.Tensor(v)
                for k, v in body_parms.items()
                if k in ['pose_body', 'root_orient', "betas"]
            }
        )
SMPLX_JointPos = c2c(body_pose.Jtr)

If I use the code as is the joints location is correctly displayed but it's not aligned with the engine I'm using, and no matter what kind of transformations I'm using I can't align it :(

A few things to note:

1. I need to transform body_pose and global_orient to euler angles otherwise human body prior won't work.
2. I have no idea what's the frame used to express the rotations in PHALP. Meaning what are the up/down, forward/backward and right/left axis and their direction. In the physics engine I'm the reference frame I'm using is +x right, -x is left, +y is up, -y is down, +z is forward and -z is backward.
3. Also human body prior needs a 'trans' attribute for the root translation otherwise the joints locations move in place, do you have any insights on how to get this value?

Any help would be very appreciated!

Cheers, -Vittorio