Bug in `get_rifke` and `get_cont6d_params` and possible solutions

[LvXinTao]

Hi Guo, thanks for your contribution to the t2m community! However, when I run your code of HumanML3D pre-processing and try to visualize the results after get_rifke function, which is :

def get_rifke(positions):

        '''Local pose, root at origin in the XZ plane'''
        positions[..., 0] -= positions[:, 0:1, 0]
        positions[..., 2] -= positions[:, 0:1, 2]

        '''All pose face Z+'''
        # # positions = qrot_np(np.repeat(r_rot[:, None], positions.shape[1], axis=1), positions) # Modified here, since we represent the root_quat as target->forward
        positions = qrot_np(np.repeat(r_rot[:, None], positions.shape[1], axis=1), positions) 

        return positions

Then something strange happens—— the result seems not to be irrelevant to global rotation( which should be facing Z+ after the function). Here's the visualiztation result:

https://github.com/user-attachments/assets/9d00c81e-f126-4a3c-bb8e-dddb90a39930

You can see that the motion is totally NOT facing Z+. After many debuggings, I found one possible solution to that, it seems in motion_representation.ipynb/get_cont6d_params

def get_cont6d_params(positions):
    skel = Skeleton(n_raw_offsets, kinematic_chain, "cpu")
    # (seq_len, joints_num, 4)
    quat_params = skel.inverse_kinematics_np(positions, face_joint_indx, smooth_forward=True)

You smooth the forward direction, which makes the global rotation inconsistent with the global positions. After setting the smooth_forward=False and visualizing the same motion, this time it seems to be on the right path! Here's the result:

https://github.com/user-attachments/assets/0c0906e0-a587-4e61-87c9-e0c5062ae7a5

I hope you can fix that and make this solution available to others!

[EricGuo5513]

Hi,

Thanks for pointing this out. I also find this operation would lead to some failures.

On Tue, Jul 30, 2024 at 3:05 AM Indigo @.***> wrote:

Hi Guo, thanks for your contribution to the t2m community! However, when I run your code of HumanML3D pre-processing and try to visualize the results after get_rifke function, which is :

def get_rifke(positions):

    '''Local pose, root at origin in the XZ plane'''
    positions[..., 0] -= positions[:, 0:1, 0]
    positions[..., 2] -= positions[:, 0:1, 2]

    '''All pose face Z+'''
    # # positions = qrot_np(np.repeat(r_rot[:, None], positions.shape[1], axis=1), positions) # Modified here, since we represent the root_quat as target->forward
    positions = qrot_np(np.repeat(r_rot[:, None], positions.shape[1], axis=1), positions)

    return positions

Then something strange happens—— the result seems not to be irrelevant to global rotation( which should be facing Z+ after the function). Here's the visualiztation result:

https://github.com/user-attachments/assets/9d00c81e-f126-4a3c-bb8e-dddb90a39930

You can see that the motion is totally NOT facing Z+. After many debuggings, I found one possible solution to that, it seems in motion_representation.ipynb/get_cont6d_params

def get_cont6d_params(positions): skel = Skeleton(n_raw_offsets, kinematic_chain, "cpu")

(seq_len, joints_num, 4)

quat_params = skel.inverse_kinematics_np(positions, face_joint_indx, smooth_forward=True)

You smooth the forward direction, which makes the global rotation inconsistent with the global positions. After setting the smooth_forward=False and visualizing the same motion, this time it seems to be on the right path! Here's the result:

https://github.com/user-attachments/assets/0c0906e0-a587-4e61-87c9-e0c5062ae7a5

I hope you can fix that and make this solution available to others!

— Reply to this email directly, view it on GitHub https://github.com/EricGuo5513/HumanML3D/issues/141, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRYNB4JQZOPNCEHCBBJN53ZO43MLAVCNFSM6AAAAABLVWZZ3CVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQZTOMBXGAZTMMI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[LvXinTao]

Thanks for your prompt reply!