search

YoungSeng / QPGesture

QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation (CVPR 2023 Highlight)
82 stars 7 forks source link

BEAT datasets #21

Open zhou431496 opened 8 months ago

zhou431496 commented 8 months ago

Hi, this is excellent work. I don't quite understand how to process the BEAT data. Is it using the full data set or just the English data set. Why matching can achieve semantic awareness.

YoungSeng commented 8 months ago

For speakers 1 and 10 of BEAT, databases were constructed separately; semantics was used with the semantic information obtained from the sentences in the 0.5s window before and after each moment after sentenceBERT, and then the cosine similarity was used to calculate the match of the text.