Issue about balance argument

[thomascong121]

Hi, Thanks for the great work, I am really interested in this idea of using contrastive learning in imbalanced studies. I am just a bit confused about this --balance term in the argument, are you just using this term to control whether to apply resampling or not?

[YMarrakchi]

Hi and thanks for the interest in our code. You are completely right. The argument --balance is meant for controlling whether resampling will be applied.

[thomascong121]

Hi, Thanks for the reply, I try to use your code to obtain the results on APTOS19. The result I got is ~0.803 which is always a bit lower than the result reported in the paper. I think this might due to the difference in randomly splitting the dataset. Or do you have any other ideas why this happens?

[YMarrakchi]

Hi. I am missing the setup you want to reproduce but either case:

• different splits lead to slightly different results but still within the ranges mentioned in the paper
• You might have noticed that the default hyperparameters match those of the ISIC-2018 dasaset. In order to get the reported performance, please double-check them to match the mentioned values in the paper.
• Similarly, the data augmentation pipeline fits the requirements of the ISIC-2018 dataset. It turns out that you need to custom your data augmentation pipeline depending on the dataset at hand. Actually moving from supervised learning methods to self-supervised learning methods lead to an increasing importance of the design of data augmentation pipelines. Although we consider here the supervised version of SimCLR, the importance of the data augmentation pipeline remains as critical as in the self-supervised scenario.
• the optimization of the data augmentation pipeline should be done for both steps i.e. you might end up using slightly different augmentation pipelines. Since the memory footprint for the second stage is way smaller, you typically consider larger crops. Also removing strong augmentations like color jittering helps a lot boosting the performance.

[thomascong121]

Hi, Thanks for the detailed explanations. In order to reproduce the results on APTOS19, I follow the same pipline as stated in the paper. For stage one (feature learning), I trained the model for 500 epochs, the learning rate is initialized to 10^−1 and decayed to 10^−4, the weight decay is set to 10^−4, the batch size is set to 192 on 4 3090 GPUs. I use RandomResizedCrop(224), RandomHorizontalFlip() and ColorJitter(brightness=0.4, contrast=0.4, saturation=0.4, hue=0.2) as the augmentation. For the second stage (classifier training), I use the same augmentation setting, but set the number of epochs equals to 100 and learning rate equals to 1.0. Can you specify if this matches with your setting on APTOS19? Much appreciates in advance.