search

auspicious3000 / SpeechSplit

Unsupervised Speech Decomposition Via Triple Information Bottleneck
http://arxiv.org/abs/2004.11284
MIT License
647 stars 92 forks source link

How to split "Accent" information of the speaker? #30

Open rppravin opened 3 years ago

rppravin commented 3 years ago

Thanks for the codebase. Good work!

In the paper, Speech is split into -- timbre (using speaker embedding), pitch, rhythm, content. If I am not wrong, the accent information of the speaker is not captured by the speaker embedding. (I know this because when I experimented with AutoVC codebase, the speaker embedding did not capture the accent info. It accent info of the source speech was always seen in the voice conversion output.)

Any ideas on how to split the accent information from speech?

Thanks, Pravin

auspicious3000 commented 3 years ago

Good question. AutoVC can disentangle accent to some extent but not sufficient. Disentangling accent is another interesting research problem. We do not have a solution for that.

rppravin commented 3 years ago

Thanks @auspicious3000

After Speech decomposition into timbre, pitch, rhythm, content -- Would you expect the accent information to be a part of the content embedding?

auspicious3000 commented 3 years ago

Accent may be a part of each component.