How to finetune the NEST model with CTC Loss for ASR task?

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html

Apache License 2.0

12.22k stars 2.54k forks source link

How to finetune the NEST model with CTC Loss for ASR task? #11163

Open zw76859420 opened 2 weeks ago

zw76859420 commented 2 weeks ago

We have trained SSL model based on the NEST, how to finetune the previous model based on the CTC Loss function? The pretraining scripts are as follows: NEST

tomschelsen commented 9 hours ago

Have you checked paragraph "Fine-tuning with pre-trained encoder" of https://github.com/NVIDIA/NeMo/blob/stable/tutorials/asr/Self_Supervised_Pre_Training.ipynb ?