search

IgorSusmelj / barlowtwins

Implementation of Barlow Twins paper
MIT License
99 stars 12 forks source link

Single GPU #3

Open Mushtaqml opened 3 years ago

Mushtaqml commented 3 years ago

If we train it using single GPU, will we face the issue of information leakage due to BN as mentioned in the MoCo paper?

Thanks

IgorSusmelj commented 3 years ago

We didn't notice any effect. The test accuracy is on par with other models like SimCLR, MoCo. You can easily introduce split batch norm using lightly.

Just change line 86 in main.py from

resnet = lightly.models.ResNetGenerator('resnet-18')

to 

resnet = lightly.models.ResNetGenerator('resnet-18', num_splits=8)

to simulate a run on a machine using 8 GPUs. See https://docs.lightly.ai/lightly.models.html for more information.

Mushtaqml commented 3 years ago

Thank you for the response.