search

gitmylo / bark-voice-cloning-HuBERT-quantizer

The code for the bark-voicecloning model. Training and inference.
MIT License
671 stars 111 forks source link

change semantic vocab size to 500 to fit hubert quantizer #21

Closed jidanhuang closed 1 year ago

jidanhuang commented 1 year ago

hi~I wonder is it possible to change semantic vocab size from 10000 to 500 to fit hubert quantizer and train semantic transformer and then coarse transformer.

gitmylo commented 1 year ago

This model is specifically meant to replicate Bark's vocab size so it can clone bark. But you can technically create a model with a different vocab size by providing a different output_size to the CustomTokenizer when creating the model.

You do not need this repository if you want to train Bark on semantic tokens with a vocab size of 500. This quantizer is specifically made to copy an existing quantizer, not make a new one, because that's just a normal quantizer in that case.