search

facebookresearch / fairseq2

FAIR Sequence Modeling Toolkit 2
https://facebookresearch.github.io/fairseq2/
MIT License
684 stars 79 forks source link

About dataset part and trainer part of fairseq2 #3

Open shanguanma opened 1 year ago

shanguanma commented 1 year ago

Until now, I don't seem to have found any details about the specific example of dataset organizing and training from scratch , Would fairseq2 intend to provide a separate trainer library and dataset library like lightning and accelerate ,datasets from hugging face.? I think that fairseq2 mainly focuses on model details like transformers library from hugging face now.

I'd like to decouple training and modeling so that it's more conducive to updating the training part quickly, one of the main pain points of self-supervised large models at the moment is that training is time-consuming and slow。 While, I hope that fairseq2 has a good recipe organized for example, maybe like speech open source projects (i.e. kaldi, espnet,wenet and so on), to solve fairseq example is unfriendly and some details missing for new users.

sidharthrajaram commented 4 months ago

Have the same issue as above ^

Are there any examples or documentation regarding how to train from scratch based on a pre-determined architecture? (Related: https://github.com/facebookresearch/fairseq2/issues/305)