cifar10_train.yaml `me_max` parameter

facebookresearch / suncet

Code to reproduce the results in the FAIR research papers "Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples" https://arxiv.org/abs/2104.13963 and "Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations" https://arxiv.org/abs/2006.10803

MIT License

486 stars 67 forks source link

cifar10_train.yaml `me_max` parameter #13

Closed jmarrietar closed 3 years ago

jmarrietar commented 3 years ago

Hi @MidoAssran, I have a question regarding the default CIFAR-10 YAML config file.

For CIFAR10 shouldn't the regularizer to be set to False? me_max: false ?. When I use True (Default parameter in cifar10_train.yaml file the loss is negative all the training).

When I put it to False it matches more with the loss reported for CIFAR-10 in the paper Screen Shot 2021-06-03 at 10 51 42 AM

MidoAssran commented 3 years ago

hi @jmarrietar , what's plotted is only the cross-entropy portion of the loss (not the loss + me-max), even though we had me_max: true. The me-max regularizer (negative entropy of average prediction) tends to stay close to its minimum value during training so it's not that interesting to look at. Turning it off may not make much of a difference, but we left it on for the CIFAR10 experiments. Hope that clarifies things!

jmarrietar commented 3 years ago

Yes, That makes Sense @MidoAssran Thank you!.