Write w2v-BERT pretraining recipe

facebookresearch / fairseq2

FAIR Sequence Modeling Toolkit 2

https://facebookresearch.github.io/fairseq2/

MIT License

613 stars 59 forks source link

Write w2v-BERT pretraining recipe #313

Open cbalioglu opened 4 months ago

cbalioglu commented 4 months ago

As the second recipe after NLLB, write the w2v-BERT (and wav2vec2) pretraining recipe for users to check out. This will likely branch to several subtasks once we start working on it.

seastar105 commented 3 months ago

@cbalioglu any progress on pre-train or finetune w2v bert recipe?

kdcyberdude commented 1 month ago

Hi @cbalioglu, Any update on this... We want to do the continual training of w2v-bert on specific Indic low-resource languages on audio-only data. Any suggestion on how should we approach this?