search

ibrahimethemhamamci / CT-CLIP

Developing Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography
197 stars 21 forks source link

Training command with multi-GPU for CT-Clip Zero-shot model #14

Closed qic999 closed 6 months ago

qic999 commented 6 months ago

Hi, thank you for your excellent work again! I would like to ask if the command for multi-GPU training is also accelerate launch run_train.py? And do you use a config for multi-GPU training?

sezginerr commented 6 months ago

We actually use FSDP for CLIP training, so the command is:

accelerate launch --use_fsdp run_train.py

However, it should theoretically work without FSDP, though I am not sure whether it would fit in the GPU memory. We did not use any config for Accelerate.

qic999 commented 6 months ago

Thanks for your quick response!

sezginerr commented 6 months ago

Hi @qic999, please see #16 if you use fsdp.