valeoai / WaffleIron

Other
39 stars 6 forks source link

can your backbone use multi-gpu to train #6

Closed mooncakehub closed 6 months ago

gpuy commented 6 months ago

Hi @mooncakehub,

Yes, you can train this backbone using multiple gpus. I have been training models using from 1 to 8 gpus with this code.

For example, on nuscenes, I am using:

python launch_train.py \
--dataset nuscenes \
--path_dataset /path/to/nuscenes/ \
--log_path ./logs/WaffleIron-48-384__nuscenes/ \
--config ./configs/WaffleIron-48-384__nuscenes.yaml \
--multiprocessing-distributed \
--fp16

The flag --multiprocessing-distributed activates multi-gpu training on all available gpus on your node. You can refer to this section of the README file for other examples.

Please also adapt the number of workers for dataloading in the config file. This parameter defines the total number of workers across all gpus: if you have 4 available GPUs, then num_workers: 12 means each gpu will be assigned 3 workers for dataloading.

Hope this helps!