search

nervjack2 / MelHuBERT

Official implementation of MelHuBERT
MIT License
64 stars 4 forks source link

How to fintune model on ASR task from stage-2? #6

Closed zw76859420 closed 1 month ago

zw76859420 commented 3 months ago

Dear authors,

Can you tell me how to finetune a model trained in stage-2 for ASR task? We can also draw a conclusion of your paper, for ASR Task, we can directly pretrain stage-2 from scratch without training stage-1?

Looking forward to your reply. Shylock

nervjack2 commented 1 month ago

Hi Shylock,

For ASR task finetuning, I suggest implementing it using s3prl or espnet. In our paper, we concluded that initializing stage-2 pre-training with stage-1 weights is not beneficial. However, training a stage-1 model is still necessary, as the pre-training target for the stage-2 model is constructed by applying k-means clustering to the features from the stage-1 model.

zw76859420 commented 1 month ago

Thanks for your reply. I understand what you mean.