bshall / hubert

HuBERT content encoders for: A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion
https://bshall.github.io/soft-vc/
MIT License
330 stars 53 forks source link

Finetuning Hubert Discrete and Soft #16

Open slimsushi opened 7 months ago

slimsushi commented 7 months ago

Hi,

thank you for your great work, it helps me a lot!

I have a question about the Hubert discrete model. Have you done any kind of training for this model too or did you just use the Hubert_base model and put a k-means clustering on top without training afterwards? Or do you have any kind of training script for the discrete model?

Since I want to finetune the Hubert soft model on a different language dataset and compare the performance with the pretrained model, I understood it the way that I would need the hubert_discrete also to be finetuned on the dataset to achieve the results I am hoping for.

Thank you for your help.

Best regards