Is it possible to fine-tune this model, or any method to update the vocabulary of it?

[ShivinM-17]

Issue Description: When utilizing the 'medium' model, it has come to my attention that certain medical terms may not be transcribed accurately. For instance: "Amoxicillin" is transcribed as "Amoxanine." "Effexor" is transcribed as "Afexa."

So, is there any possibility of fine-tuning the 'medium' model to enhance its capability to detect and transcribe such words more accurately?

Can someone provide a sample/reference code if there is any possibility?

[remic33]

You can finetune faster_whisper models. Just fine tune a regular whisper model and you will be able to pass those weigth to faster_whisper model.

[davidlandais]

Fine-tune the standard whisper model. Here is an entry article: https://huggingface.co/blog/fine-tune-whisper Since faster-whisper is using ctranslate2 you will need to concert your model: https://opennmt.net/CTranslate2/guides/transformers.html#whisper Because WhisperX is using faster_whisper, you should use a translated model. So here is the chain.

[0xm00n]

Fine-tune the standard whisper model. Here is an entry article: https://huggingface.co/blog/fine-tune-whisper Since faster-whisper is using ctranslate2 you will need to concert your model: https://opennmt.net/CTranslate2/guides/transformers.html#whisper Because WhisperX is using faster_whisper, you should use a translated model. So here is the chain.

So I just need to convert a standard whisper model using ctranslate2 and pass the resulting model to WhisperX?