search

youngwoo-yoon / Co-Speech_Gesture_Generation

This is an implementation of Robots learn social skills: End-to-end learning of co-speech gesture generation for humanoid robots.
https://sites.google.com/view/youngwoo-yoon/projects/co-speech-gesture-generation
Other
72 stars 9 forks source link

No audio input #7

Closed birdflies closed 2 years ago

birdflies commented 3 years ago

Dear, This project only support text input. It seems that you did not use audio data~ 

 in_text = in_text.to(device)
 in_audio = in_audio.to(device)
 target_vec = target_vec.to(device)
 loss = train_iter_seq2seq(args, epoch, in_text, text_lengths, target_vec, generator, gen_optimizer)
youngwoo-yoon commented 3 years ago

You're right. This project do not use audio data so in_audio is never used. If you're interested in using text+audio data, please see our follow-up project https://github.com/ai4r/Gesture-Generation-from-Trimodal-Context.

birdflies commented 3 years ago

Thanks, do Gesture-Generation-from-Trimodal-Context support the input format of Trinity Speech-Gesture Dataset for GENEA Challenge 2020?

youngwoo-yoon commented 3 years ago

No, it doesn't. You have to modify the code to make it compatible with the GENEA challenge dataset.