search

fishaudio / fish-diffusion

An easy to understand TTS / SVS / SVC framework
https://diff.fish.audio
MIT License
635 stars 81 forks source link

About pitch extract on multi speaker or singer database #112

Closed wblgers closed 1 year ago

wblgers commented 1 year ago

Hi,

I want to train a multi speaker VC or SVC model based on fish diffusion. However, I noticed that there is no speaker-related f0 mean or std stored in the preprocess pipeline. But the f0 range of female and male singer are different, they share the same normalize (pitch_to_scale), is there any problem on multi singer training? Or How to resolve this problem on fish diffusion?

Thanks!

Stardust-minus commented 1 year ago

That was not a problem.The f0 analysis range is from 20 to 16,000Hz.It covers all the range which human can produce.

wblgers commented 1 year ago

16,000Hz

I notice that in BasePitchExtractor, the f0_min is 50, the f0_max is 1100.

Stardust-minus commented 1 year ago

Oh,I have the mistake.Normally I use the configure that f0_min=20Hz,f0_max=1600Hz. Under normal circumstances, f0 analyzers are sufficient for analyzing these frequency bands (most of the human vocal frequencies are in this range). Higher frequency analysis is meaningless, and neither the acoustic model nor the f0 analyzer can handle it.

wblgers commented 1 year ago

OK, thanks for your info. And, if I want to use this f0 process on expressive speech(like the dialogue in a tvplay), not singing, can the f0 in fish diffusion handle this scene where there is roaring now and then?

Stardust-minus commented 1 year ago

OK, thanks for your info. And, if I want to use this f0 process on expressive speech(like the dialogue in a tvplay), not singing, can the f0 in fish diffusion handle this scene where there is roaring now and then?

There is multi f0 analyzers,the best is crepe but it is pretty slow.You can try these analyzers (pm,dio,harvest,crepe)and find the f0analyzers can handle it.But sometimes for some extreme audio, no f0 analyzer can accurately extract, so good luck.:)

wblgers commented 1 year ago

OK, thanks for your info. And, if I want to use this f0 process on expressive speech(like the dialogue in a tvplay), not singing, can the f0 in fish diffusion handle this scene where there is roaring now and then?

There is multi f0 analyzers,the best is crepe but it is pretty slow.You can try these analyzers (pm,dio,harvest,crepe)and find the f0analyzers can handle it.But sometimes for some extreme audio, no f0 analyzer can accurately extract, so good luck.:)

OK , thanks!