How to use multiple GPUs for finetune?

rmaphoh / RETFound_MAE

RETFound - A foundation model for retinal image

Other

311 stars 63 forks source link

How to use multiple GPUs for finetune? #4

Closed Akemimadokami closed 9 months ago

Akemimadokami commented 9 months ago

I am a greenhand at coding, wondering how to use multiple GPUs for finetune? I have 4 available GPUs and have set "--world_size" to 4, but it seems there is only 1 GPU being used.

rmaphoh commented 9 months ago

Hi there, for multiple GPU training please change "n_proc_per_node" and "world_size" to 4 (4 GPUs as an example).

Akemimadokami commented 9 months ago

Thanks a lot!