search

ZiqiaoPeng / SyncTalk

[CVPR 2024] This is the official source for our paper "SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis"
https://ziqiaopeng.github.io/synctalk/
Other
892 stars 92 forks source link

Real-time inference #55

Open a-ghorbani opened 1 month ago

a-ghorbani commented 1 month ago

Is SyncTalk suitable for real-time inference? Are there any stats about latency anf performance? Are there any benchmarks or optimization tips for real-time use?

oguzhandoganoglu commented 1 week ago

Good question !!

jinqiupeter commented 1 week ago

It definitely can. On my RTX 3090 I can have ~50 fps if everything is loaded to GPU. Here is my example of realtime chatting using SyncTalk:

https://github.com/ZiqiaoPeng/SyncTalk/assets/12045814/623e5309-7d71-4fab-ad41-2280566b422a

oguzhandoganoglu commented 1 week ago

How do you send a request after training the model? Can you share the code? Is it in the form of a test in the documentation?

jinqiupeter commented 1 week ago

Lipsync is also very good:

https://github.com/ZiqiaoPeng/SyncTalk/assets/12045814/67bb8404-3f34-4363-b260-adcba5fefe3b

oguzhandoganoglu commented 1 week ago

Video doesn’t work. Alsı could you please answer my question if possible

oguzhandoganoglu commented 1 week ago

How do you send a request after training the model? Can you share the code? Is it in the form of a test in the documentation? @jinqiupeter