search

gitmylo / bark-voice-cloning-HuBERT-quantizer

The code for the bark-voicecloning model. Training and inference.
MIT License
671 stars 111 forks source link

semantic.npy #7

Closed taalua closed 1 year ago

taalua commented 1 year ago

Hi, thank you for the work. In your code, I understand that Hubert is used to extract: feat_string = '_semantic_features.npy' how about:
sem_string = '_semantic.npy' where to get the *_semantic.npy?

Thank you.

gitmylo commented 1 year ago

Those are the files for training a custom quantizer. they can be created with https://github.com/gitmylo/bark-data-gen

(For the record, since a lot of people seem to misunderstand, you do not need that code for cloning a voice itsself, only if you want to train your own model (which usually isn't needed))

taalua commented 1 year ago

Thank you for your reply. I checked https://github.com/gitmylo/bark-data-gen is the *_semantic.npy is created using create_data.py ?

gitmylo commented 1 year ago

correct, but they will have random names, first you run create_data, then when you have enough semantics, you run create_wavs. wait until that's done. then you can zip both folders, place them in a directory /semantic and /wav.

Then do the process and process2 steps from this repo. after that, you can train your own custom quantizer.