search

YuanxunLu / LiveSpeechPortraits

Live Speech Portraits: Real-Time Photorealistic Talking-Head Animation (SIGGRAPH Asia 2021)
MIT License
1.22k stars 210 forks source link

Streaming audio data inference #41

Open rosklyar opened 2 years ago

rosklyar commented 2 years ago

Hey there! Many thanks for your work - it looks awesome! I tried to implement LIVE Audio2Headpose only pipeline based on your work and on inferencing chunks I see big gaps in result values between end of previous chunks and start of next one(APC_model is using historic data). So the curve is not smooth. What is the best way to solve this issue? Can inferencing with sliding window help?

YuanxunLu commented 2 years ago

Smoothing on the results helps.

deema-A commented 1 year ago

@rosklyar @YuanxunLu may you please give some instructions/hints on how to make this streamable? Thanx!

smtabatabaie commented 1 year ago

@rosklyar @YuanxunLu may you please give some instructions/hints on how to make this streamable? Thanx!

Would really help, I'm struggling to see how it is working in real-time.

cjw-d commented 1 year ago

@YuanxunLu I'm also struggling to make it streamable.Would you please give some hints?Thanks!