Can it be used to solve the unbalanced problem of supervised learning? And How?

[Rayzlx]

Thanks for sharing the codes! Hope you can answer my question！

[YyzHarry]

Hi, thanks for your interest. The techniques are indeed designed in the context of supervised learning --- we have labels, but they are imbalanced. To apply:

• Semi-supervised: You just need to first collect (relevant) unlabeled data. As indicated by our results, as long as the data relevance is high, even the unlabeled data is also (highly) imbalanced, it can still (almost always) improve the imbalanced learning results. Then you can just use self-training, or other semi-supervised techniques (advanced methods such as VAT/MT will achieve better results, see our Appendix E.1).
• Self-supervised: Depending on your data format, you can choose any self-supervised learning methods that are suitable for your own data. All you need is to (1) first do self-supervised pre-training without using imbalanced labels; and (2) then you can use any base training method to do supervised training. As our paper shows, it can consistently improve the results, regardless of the base training techniques.

For more details please refer to our paper.

[YuemingJin]

Hi,

Thanks for your great work! I am wondering that in the self-supervised setting, in the step (2) of supervised training, do you fix the backbone parameter (training on the step (1)) about representation learning? Or you fine-tune all the parameters of the whole network? Thanks!!

[YyzHarry]

Hi @YuemingJin - thanks for your interest. We did not fix the backbone parameter during the supervised training stage. Different from linear evaluation protocol in self-supervised learning, we here want to maximize the performance for an imbalanced learning task, hence we do not fix the parameters.