search

SJTMusicTeam / Muskits

An opensource music processing toolkit
Apache License 2.0
312 stars 44 forks source link

note dur and note pitch embedding is added with phone embedding or concat with embedding #120

Closed 11721206 closed 2 years ago

11721206 commented 2 years ago

Hello, I just want to know note dur and note pitch embedding is added with phone embedding or concat with embedding? my experiment show concat result is very bad when i change my note pitch. Thank You

ftshijt commented 2 years ago

Hi, Thanks for raising the question. Generally speaking, it depends on the models.

For Xiaoice, the embedding is added up as in https://github.com/SJTMusicTeam/Muskits/blob/3750a54f414761bb8bd72b1f0914b442d34cb176/muskit/svs/xiaoice/XiaoiceSing.py#L447 For transformer-based SVS, the embedding is either concat or added up as in https://github.com/SJTMusicTeam/Muskits/blob/3750a54f414761bb8bd72b1f0914b442d34cb176/muskit/svs/glu_transformer/glu_transformer.py#L758-L761 For RNN-based model, the embedding is either concat or added up as in https://github.com/SJTMusicTeam/Muskits/blob/3750a54f414761bb8bd72b1f0914b442d34cb176/muskit/svs/naive_rnn/naive_rnn.py#L445-L450

ftshijt commented 2 years ago

Could you refer to which recipe you are working with so that we could check and even help you investigate the issues?

ftshijt commented 2 years ago

Since there is no response for a long time, I will close the issue