search

facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.17k stars 6.37k forks source link

Wav2Vec2 Pretraining #5533

Open rajeevbaalwan opened 1 month ago

rajeevbaalwan commented 1 month ago

❓ Questions and Help

I want to perform wav2vec2 Pretraining from scratch and while following the documentation for same on https://github.com/facebookresearch/fairseq/tree/main/examples/wav2vec it is mentioned that all audio clips should be in single directory. The issue is i have too much data to keep in a single directory.

I have data in multiple directories on different disks and can't move complete data in single directory due to storage issue. Is it possible to pretrain the model in this scenario?

gau-nernst commented 2 weeks ago

You can try using symlinks

rajeevbaalwan commented 2 weeks ago

@gau-nernst thanks for your response. Is there any other way for this like modifying the code so it can handle paths from multiple directories? At the end a path is to be loaded by the data loader so why this constraint?

gau-nernst commented 1 week ago

Of course it is possible to modify the code, but then you have to do it yourself.