search

rhgao / co-separation

Co-Separating Sounds of Visual Objects (ICCV 2019)
Creative Commons Attribution 4.0 International
92 stars 23 forks source link

Audio visual speech separation #7

Closed 83344rushikesh closed 4 years ago

83344rushikesh commented 4 years ago

@rhgao Will it work on videos which contains multiple speakers to isolate them in multiple files equal to the number of the speakers present in the videos?

rhgao commented 4 years ago

The current model is frame-based (no motion) so it cannot handle instance separation such as speech separation. It would be interesting future work to incorporate motion to separate speech for multiple speakers.