search

facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.38k stars 6.4k forks source link

can not use multi-gpu in data2vec training #4226

Open synsis opened 2 years ago

synsis commented 2 years ago

I use this code

code------------------ python fairseq_cli/hydra_train.py -m --config-dir examples/data2vec/config/audio/pretraining \ --config-name base_librispeech task.data=/path/to/manifests common.user_dir=examples/data2vec code------------------

to train data2vec model, distributed_training.distributed_world_size is set to 16, but, the log and the system shows there are only one gpu used in training.

log------------------ [2022-02-18 08:45:58,583][fairseq_cli.train][INFO] - training on 1 devices (GPUs/TPUs) log------------------

usuyama commented 1 year ago

you might need to check WORLD_SIZE env var