search

open-mmlab / mmselfsup

OpenMMLab Self-Supervised Learning Toolbox and Benchmark
https://mmselfsup.readthedocs.io/en/latest/
Apache License 2.0
3.14k stars 429 forks source link

Default Setting of # of nodes & GPUs #753

Closed Divadi closed 1 year ago

Divadi commented 1 year ago

What is the default # of nodes & # of GPUs per node for training? I believed it to be 8 nodes, each 8 GPUs but, I just wanted to double check since "8x512" led me to believe it was perhaps 8 GPUs with 512 batch size each.

I apologize if this has been answered elsewhere.

fangyixiao18 commented 1 year ago

What is the default # of nodes & # of GPUs per node for training? I believed it to be 8 nodes, each 8 GPUs but, I just wanted to double check since "8x512" led me to believe it was perhaps 8 GPUs with 512 batch size each.

I apologize if this has been answered elsewhere.

Yes, the total batch size is 8 x 512 = 4096

Divadi commented 1 year ago

Thank you!