mosaicml / examples

Fast and flexible reference benchmarks
Apache License 2.0
441 stars 124 forks source link

Confusing comment in the deeplabv3.yaml #59

Open Godofnothing opened 1 year ago

Godofnothing commented 1 year ago

In the example of deeplab training in the yaml config file deeplab/yamls/deeplabv3.yaml it is written in config file:

batch_size: 128                    # Training dataloader batch size per device - line 26
batch_size: 128                    # Evaluation dataloader batch size per device - line 39

But, according to the main.py (lines 64-66) it should be not the batch size per device rather the total batch size.

if dist.get_world_size():
    train_batch_size //= dist.get_world_size()
    eval_batch_size //= dist.get_world_size()
Landanjs commented 1 year ago

Yes, that comment is incorrect. batch_size is the total batch size not per device batch size. We will fix it ASAP, thank you for the notice!