search

Mael-zys / T2M-GPT

(CVPR 2023) Pytorch implementation of “T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations”
https://mael-zys.github.io/T2M-GPT/
Apache License 2.0
574 stars 52 forks source link

Constraining starting and ending pose #56

Closed LeyangWen closed 11 months ago

LeyangWen commented 1 year ago

Hi,

Your work is really interesting and the results look great. Is there a way to enforce a specific starting & ending human pose using your network? That way, you can ask the generated pose to do specific tasks (e.g., moving from location A to location B) and even link multiple generated sequences together (instead of using "and" in the text prompt).

I am thinking of modifying the loss function (i.e., penalizing the pose difference at the starting and ending frame) and retraining the network. Would it be the best way to accomplish this?

Jiro-zhang commented 12 months ago

Thank you for your interest in our work and your great question.

Our work is not currently able to do temporal splicing without additional training. However, we are looking into this area to address this problem with specialized training.

Constraining by loss functions I personally think is a potentially effective solution. On the basis of your proposed method, I think it is necessary to introduce an additional motion length into the gpt as a signal.