how to apply multi-GPU training

[didadida-r]

hi, i am runing the CALP project but i dont know how to run it in multi-GPU mode in single node. Given the script in train-only-clotho.sh, can you give some guidance on running CALP project in multi-GPU mode on a single node thanks

python -m laion_clap.training.main \
    --save-frequency 5 \
    --save-top-performance 3 \
    --save-most-recent \
    --dataset-type="webdataset" \
    --datasetpath="${dataset_path}" \
    --precision="fp32" \
    --batch-size=4 \
    --lr=1e-4 \
    --wd=0.0 \
    --epochs=45 \
    --report-to "tensorboard" \
    --workers=6 \
    --use-bn-sync \
    --amodel HTSAT-tiny \
    --tmodel roberta \
    --warmup 3200 \
    --datasetnames "Clotho" \
    --top-k-checkpoint-select-dataset="Clotho-test" \
    --top-k-checkpoint-select-metric="mAP@10" \
    --logs "${exp_dir}" \
    --seed 3407 \
    --gather-with-grad \
    --optimizer "adam" \
    --data-filling "repeatpad" \
    --data-truncating "rand_trunc" \
    --pretrained-audio "$pretrained_audio_model" \
    --prefetch-factor 2

[Sreyan88]

Hi @didadida-r , were you able to figure out how to do this? Thanks!