When I train the model with 4 GPUs, should I set the sync_bn to true?

[rockywind]

Thank you for you help.

parser.add_argument('--sync_bn', action='store_true', default=False, help='whether to use sync bn')

[codyreading]

Yup, I set it as True when doing multi GPU training

[rockywind]

Thank you for your help! But,when I can't set the sync_bn as True, when I doing multi GPU inference.

[codyreading]

Yes, this only affects training, as this calculates the Batch Norm mean and variance across the full batch (all GPUs) when training. Inference these parameters are fixed, so you don't need to set anything there. Please see here

[rockywind]

Thank you very much!