search

jasonppy / VoiceCraft

Zero-Shot Speech Editing and Text-to-Speech in the Wild
Other
7.52k stars 739 forks source link

non-speech sounds like suno bark ? #61

Open Revln9 opened 6 months ago

Revln9 commented 6 months ago

is it possible to generate interjections with this model like mhmm or aha ?

jasonppy commented 6 months ago

Thanks! There are two possible way to have interjections:

  1. If the prompt has sounds like that, it's possible that the model will add that to the generation spontaneously
  2. if you wrote these in the target transcript, the model will just follow
yoesak commented 5 months ago

Thanks! There are two possible way to have interjections:

  1. If the prompt has sounds like that, it's possible that the model will add that to the generation spontaneously
  2. if you wrote these in the target transcript, the model will just follow

So theoretically, VoiceCraft can be used to generate singing melody (vocal only) ?

jasonppy commented 5 months ago

Thanks! There are two possible way to have interjections:

  1. If the prompt has sounds like that, it's possible that the model will add that to the generation spontaneously
  2. if you wrote these in the target transcript, the model will just follow

So theoretically, VoiceCraft can be used to generate singing melody (vocal only) ?

I think it's unlikely to generate singing voice out-of-the-box, just because the training data is speech data. You could finetune it on singing data

yoesak commented 5 months ago

Cool, I will try with singing data. Thanks

Magauiya commented 1 month ago

Hi @yoesak! It is quite interesting to see VC for different application. Can you share your progress on generating songs using Voicecraft?