microsoft / SoftTeacher

Semi-Supervised Learning, Object Detection, ICCV2021
MIT License
900 stars 123 forks source link

How to change batch size of training? #196

Open vefak opened 2 years ago

vefak commented 2 years ago

Hello, I cannot figure out how to change batch size? I have only one GPU. I think batch size is related to samples and workers. what is the equation between them and batch size? Can somebody explain it to me, please?

samples_per_gpu=5,
 workers_per_gpu=5,
PeterVennerstrom commented 2 years ago

Batch size is samples_per_gpu * # of gpus.

Samples_per_gpu is split into labeled and unlabeled by the sample_ratio.

From the paper:

Since the amount of training data of Partially La- beled Data setting and Full Labeled Data setting has large differences, the training parameters under the two settings are slightly different. Partially Labeled Data: The model is trained for 180k it- erations on 8 GPUs with 5 image per GPU. With SGD train- ing, the learning rate is initialized to 0.01 and is divided by 10 at 110k iteration and 160k iteration. The weight decay and the momentum are set to 0.0001 and 0.9, respectively. The foreground threshold is set to 0.9 and the data sampling ratio sr is set to 0.2 and gradually decreases to 0 over the last 10k iterations. Fully Labeled Data: The model is trained for 720k itera- tions on 8 GPUs with 8 image per GPU. In SGD training, the learning rate is initialized to 0.01 and is divided by 10 at 480k iteration and 680k iteration. The weight decay and the momentum are set to 0.0001 and 0.9, respectively. The foreground threshold is set to 0.9 and the data sampling ra- tio sr is set to 0.5 and gradually decreases to 0 in the last 20k iterations.