search

vturrisi / solo-learn

solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
MIT License
1.42k stars 185 forks source link

Pretrain BYOL and SwAV -WandB problem #364

Closed kalikhademi closed 1 year ago

kalikhademi commented 1 year ago

Hi,

I am using main_pretrain.py to train resnet 50 backbones with BYOL and SwAV method using unlabeled OpenImages dataset. After finishing the runs, SwAV wandb shows a chart for train_swav_loss_step but there is not such chart for BYOL. I would like to have both of them. Could you please help me to make this change

vturrisi commented 1 year ago

Hey, The losses for the two methods are different, so they are logged into separate plots. BYOL's loss is logged as train_neg_cos_sim.

You can change it here https://github.com/vturrisi/solo-learn/blob/d27c7130d19035c0ba0af8f90217e78d8ebe7f48/solo/methods/byol.py#L196-L199 Keep in mind that the scale of the losses is completely different.