search

UttaranB127 / speech2affective_gestures

This is the official implementation of the paper "Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning".
https://gamma.umd.edu/s2ag/
MIT License
44 stars 9 forks source link

Can we use this to generate pose from text? #23

Open gutzcha opened 11 months ago

gutzcha commented 11 months ago

Hello and thank you for you work, this looks really good. I was able to download and install the repo but I was unsure on how to use it, Can we create gestures from text using this model? if so, then how?

UttaranB127 commented 11 months ago

The model uses speech audio and text transcripts of the speech to generate motions. Do you mean generation from text prompts? For that, the text encoder module needs to be repurposed to work with the prompts rather than transcripts.

gutzcha commented 11 months ago

Interesting, thank you for taking the tine to answer. I will try it using a text to speach generator.