We provide a PyTorch implementation of the paper Voice Separation with an Unknown Number of Multiple Speakers In which, we present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously. The new method employs gated neural networks that are trained to separate the voices at multiple processing steps, while maintaining the speaker in each output channel fixed. A different model is trained for every number of possible speakers, and the model with the largest number of speakers is employed to select the actual number of speakers in a given sample. Our method greatly outperforms the current state of the art, which, as we show, is not competitive for more than two speakers.
Other
1.23k
stars
178
forks
source link
Transfer Learning/ Improving performance of the base model #89
Hi, I have an idea to make a svoice for specific domains, i.e medicine-related conversations, financial conversations, or maybe a different language, etc.
I am training the base model using librimix and it works pretty well. but sometimes when I test with datasets from a different language the results are not that great.
I trained the base model with librimix up to 100 epochs.
And I further train the same model with a small dataset of different languages and continue training for 20~30 epochs, I hoped it would improve the model performance on the other language while keeping at least the original performance. But unfortunately, It did not work like that. It's like the model reset all its weights when I change the dataset for training.
my question is how would I go about transfer learning so that the base model does not lose its performance but improves based on the custom data provided?
Hi, I have an idea to make a svoice for specific domains, i.e medicine-related conversations, financial conversations, or maybe a different language, etc. I am training the base model using librimix and it works pretty well. but sometimes when I test with datasets from a different language the results are not that great.
I trained the base model with librimix up to 100 epochs. And I further train the same model with a small dataset of different languages and continue training for 20~30 epochs, I hoped it would improve the model performance on the other language while keeping at least the original performance. But unfortunately, It did not work like that. It's like the model reset all its weights when I change the dataset for training.
my question is how would I go about transfer learning so that the base model does not lose its performance but improves based on the custom data provided?
Thank you