In SSL，the number of the logits have multiplied 4. Is it necessary？

[ChenJunzhi-buaa]

Thanks for your great job at first! In SSL，the number of the logits have multiplied 4. Is it necessary？ What if we just augment the data without multipling the number of the logits by 4 ?

[Impression2805]

Thank you for your interest in our paper. We use rotation based SSL and treat the rotated images as new classes (i.e., multipling the number of the logits by 4). Actually, when using rotation as a standard data augmentation, the rotation angle $\theta$ is typically small (e.g., $-30 < \theta < +30$ in [RandAugment] (https://github.com/ildoonet/pytorch-randaugment). Differently, in our method, the rotation degree is very strong: $\theta \in {90, 180, 270}$. We find that using such strong rotation degrees as data augmentation is harmful to the classification performance. For example, in the standard (not incremental) CIFAR-100 classification experiment, a ResNet-18 model can get 76.46\% accuracy, while using strong rotation based data augmentation leads to significantly lower (74.43\%) results.

Intuitively, by keeping images with very large augmentations in the same class, we may accidentally encourage models to learn overly strong invariances which do not naturally exist in the data. Therefore, treating rotated images as new classes is more reasonable when the rotation angle is large.

[ChenJunzhi-buaa]

thank you very much for your detailed reply！

---- Replied Message ---- From Date 07/12/2022 17:29 To Cc 、 Subject Re: [Impression2805/CVPR21_PASS] In SSL，the number of the logits have multiplied 4. Is it necessary？  (Issue #4) Thank you for your interest in our paper. We use rotation based SSL and treat the rotated images as new classes (i.e., multipling the number of the logits by 4). Actually, when using rotation as a standard data augmentation, the rotation angle $\theta$ is typically small (e.g., $-30 < \theta < +30$ in [RandAugment] (https://github.com/ildoonet/pytorch-randaugment). Differently, in our method, the rotation degree is very strong: $\theta \in {90, 180, 270}$. We find that using such strong rotation degrees as data augmentation is harmful to the classification performance. For example, in the standard (not incremental) CIFAR-100 classification experiment, a ResNet-18 model can get 76.46% accuracy, while using strong rotation based data augmentation leads to significantly lower (74.43%) results. Intuitively, by keeping images with very large augmentations in the same class, we may accidentally encourage models to learn overly strong invariances which do not naturally exist in the data. Therefore, treating rotated images as new classes is more reasonable when the rotation angle is large. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: <Impression2805/CVPR21_PASS/issues/4/1181536351@github.com> © Githubissues. Githubissues is a development platform for aggregating issues.