Here is the PR for audio resynthesis in discrete token:
1) We extend hubert_voc1 to token_voc1 and it can handle more models token;
2) We add f0 for training and inference when finding poor prounciation in singing;
2) We add multi-stream method including residual cluster and weight sum;
3) Using embedding feature of models is also allowed.
The following models have been validated in opencpop recipe: HuBERT, XLS-R, WavLM, MERT, Encodec.
Here is the PR for audio resynthesis in discrete token: 1) We extend hubert_voc1 to token_voc1 and it can handle more models token; 2) We add f0 for training and inference when finding poor prounciation in singing; 2) We add multi-stream method including residual cluster and weight sum; 3) Using embedding feature of models is also allowed.
The following models have been validated in opencpop recipe: HuBERT, XLS-R, WavLM, MERT, Encodec.