Training Kit-ML encountered dimension mismatch problem

OpenMotionLab / MotionGPT

[NeurIPS 2023] MotionGPT: Human Motion as a Foreign Language, a unified motion-language generation model using LLMs

https://motion-gpt.github.io

MIT License

1.51k stars 95 forks source link

Training Kit-ML encountered dimension mismatch problem #82

Open lixiang927047 opened 9 months ago

lixiang927047 commented 9 months ago

@ChenFengYe I want to use the framework of MotionGPT to train Kit-ML, but I have encountered a lot of dimensional incompatibility problems, most of which are due to the fact that the data set is only 251 dimensional and the network is set to 263 dimensional (probably because the number of summary joints in the Kit-ML data set is 21). The number of joints for HumanML3D is 22). What do I need to do with the Kit-ML data set before training? Or do different Settings in the training script?

billl-jiang commented 7 months ago

Hello,

For training Kit-ML with MotionGPT and addressing the dimension mismatch issue, you simply need to update the configuration. https://github.com/OpenMotionLab/MotionGPT/blob/a56b09ff7d20b4ddde9c0080aeb4a6b237f031c3/configs/config_h3d_stage1.yaml#L33-L34 Change mGPT.data.HumanML3D.HumanML3DDataModule to mGPT.data.Kit.KitDataModule in the config file. This should align the dimensions by adapting to the Kit dataset's structure.