search

OpenMotionLab / MotionGPT

[NeurIPS 2023] MotionGPT: Human Motion as a Foreign Language, a unified motion-language generation model using LLMs
https://motion-gpt.github.io
MIT License
1.38k stars 85 forks source link

Training on custom Dataset #92

Open Yashaswini-Srirangarajan opened 3 months ago

Yashaswini-Srirangarajan commented 3 months ago

How to process a custom MoCap dataset to a format that MotionGPT understands? Does it have to be taken through HumanML3D and AMASS pipelines to make it compliant with the existing training data structure? Thanks !!

OrigamiStationery commented 2 months ago

I also tried to train MotionGPT on my custom dataset. It seems that the motions should be transformed to HumanML3D form, which takes the first 22 joints in SMPL-form motions. Then they can be converted to motion features with 263 dimensions. Also, all the texts should follow HumanML3D form. What's more, the dataset might be placed at 'dataset/HumanML3D'. I encountered a problem that the metrics, espacially FID, couldn't be correctly calculated when the dataset is not at 'dataset/HumanML3D'.

Yashaswini-Srirangarajan commented 2 months ago

@OrigamiStationery Thanks for the reply! How did you convert your custom motion into HumanML3D format/SMPL format? Also what format was your motion capture data stored in? I have my data stored as a .trc file and that is kind of limiting me from converting it into HumanML3D or SMPL formats. Any approach to convert the data to HumanML3D format would be really helpful. Thanks!!

OrigamiStationery commented 2 months ago

@OrigamiStationery Thanks for the reply! How did you convert your custom motion into HumanML3D format/SMPL format? Also what format was your motion capture data stored in? I have my data stored as a .trc file and that is kind of limiting me from converting it into HumanML3D or SMPL formats. Any approach to convert the data to HumanML3D format would be really helpful. Thanks!!

maybe you can resort to this wonderful repo https://github.com/Dou-Yiming/Pose_to_SMPL. It can convert motion datas represented by joints position .npy file with the format of (nframes, joints_num,3), to SMPL format. As for your own dataset, you may need to define your own kinematic chain in order to use the aforementioned repo.