Open Mushtaqml opened 3 years ago
We didn't notice any effect. The test accuracy is on par with other models like SimCLR, MoCo. You can easily introduce split batch norm using lightly.
Just change line 86 in main.py
from
resnet = lightly.models.ResNetGenerator('resnet-18')
to
resnet = lightly.models.ResNetGenerator('resnet-18', num_splits=8)
to simulate a run on a machine using 8 GPUs. See https://docs.lightly.ai/lightly.models.html for more information.
Thank you for the response.
If we train it using single GPU, will we face the issue of information leakage due to BN as mentioned in the MoCo paper?
Thanks