search

mingyuan-zhang / MotionDiffuse

MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
https://mingyuan-zhang.github.io/projects/MotionDiffuse.html
Other
850 stars 74 forks source link

Waht is "xf_proj" for? #10

Closed JeremyCJM closed 1 year ago

JeremyCJM commented 2 years ago

Hi Mingyuan,

I am wondering what is "xf_proj" for in https://github.com/mingyuan-zhang/MotionDiffuse/blob/main/text2motion/models/transformer.py#L394.

Why do you select the word with the maximum embedding value?

Thanks, Jeremy

mingyuan-zhang commented 2 years ago

Hi, xf_proj is an overall semantic feature to represent the given language description. I think it's a common choice in NLP and Motion Generation that use the last token to represent the overall characteristics. This is also used in GLIDE, a typical text-to-image generative model.

JeremyCJM commented 2 years ago

Got it, thanks for your reply!