Unseen Male to Male results in Female output

[bharaniyv]

Hi, I really liked this great project and while testing it with some English male as source and Malayalam male as target sample the output sounds like a female voice, I think it may have something to do with language difference and Speaker encoder, have you come across any such scenarios and if so can you suggest anything to correct them?

Thanks

[ballerburg9005]

I found that the model mostly gravitates towards a generic male and a generic female voice, and then only transfers certain nuances it can understand from the target onto the generic voice. It seems to me that your target sample could be rather high pitch, which makes it switch to the generic female voice. And you could try to fix this by pitch-shifting your target sample. I found that even doing ridiculous pitch-shifts doesn't actually transfer to the output.